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TRIBUTE 


Tribute: 

Dr.Jacqueline  R. 
Henningsen 

Staff,  Office  of  Air  Force  Studies  and  Analysis  (A9) 


On  October  6,  2014,  the 

Air  Force  retired  the  most 
senior  and  certainly  one 
of  the  classiest  operations 
research  analysts  to  ever 
walk  the  fourth  floor  E-Ring 
at  the  Pentagon.  To  those  who  know 
her  as  Jackie,  and  the  rest  of  us 
who  know  her  as  Dr.  Henningsen, 
the  distinguished  government  career 
of  a  professional  icon  and  a  great 
servant  to  our  country  came  to  an 
end.  The  retirement  of  Dr.  Jacqueline 
R.  Henningsen,  PhD,  SES,  as  the 
Director  of  Air  Force  Studies  & 
Analyses,  Assessments,  and  Lessons 
Learned  signals  an  end  to  an  era  of 
analytic  leaders  who  grew  up  under 
the  institutional  legacy  and  teaching 
of  analytic  community  greats  such 
as  Lieutenant  General  Glenn  Kent 
and  Mr.  Clayton  Thomas.  We  will 
miss  her  dearly  for  what  she  meant 
to  us  as  coach,  mentor,  teacher,  and 
friend  and  we  look  forward  to  the  role 
she  will  continue  to  play  as  a  MORS 
Fellow  and  senior  analyst  emeritus  as 
she  enters  a  new  phase  of  her  life. 

Dr.  Henningsen  began  her  career  in 
the  Air  Force  after  nearly  two  decades 
in  education.  Her  path  may  have 
been  ordained  as  her  first  years  of  life 
were  spent  listening  to  the  rumbling 
“sound  of  freedom”  from  the  runways 


of  Langley  Air  Force  Base,  Virginia, 
and  to  the  earth-shaking  wet-take-off 
jet  engine  roar  of  the  SAC  bombers  at 
Offutt  Air  Force  Base,  Nebraska.  From 
these  front-line  Air  Force  bases  to  her 
bustling  E-Ring  office  at  the  Pentagon, 
she  has  poured  her  heart,  soul,  and 
her  passion  into  her  work.  As  the 
Air  Force  moves  through  its  seventh 
decade  as  a  separate  service,  the 
careers  of  countless  men  and  women 
in  the  analytic  and  scientific  career 
fields  have  been  touched  by  her  wise 
council  who  will  forever  be  in  her  debt. 

During  her  career  on  the  Air  Staff 
she  has  served  six  USAF  Chiefs 
of  Staff,  including  Generals  Mike 
Ryan,  John  Jumper,  Mike  Moseley, 
Norton  Schwartz,  and  Mark  Welsh. 
Supporting  and  working  with  these 
Air  Force  leaders  required  brains  as 
well  as  what  Sheryl  Sandberg  termed, 
the  ability  to  “Lean  In.”  Her  skills 
at  basketball  and  volleyball  might 
have  also  given  her  the  necessary 
“height”  to  stand  toe  to  toe  with  these 
esteemed  general  officers  whom  she 
also  calls  friends.  Running  in  these 
high  echelons  of  air  power  required 
being  accepted  into  a  very  exclusive 
network  . . .  without  a  call  sign. 

Always  “speaking  truth -to- power” 
requires  the  highest  ethical  character. 


On  a  daily  basis  Dr.  Henningsen  would 
help  senior  leadership  “fireproof”  their 
decisions  with  a  solid  analytic  body 
of  work,  and  when  they  couldn’t  be 
defended  by  solid  analytics  she  would 
calmly  say,  “There  are  many  factors 
that  go  into  a  decision  and  analysis 
is  only  one,  so  I  need  to  advise  you 
that  I  can’t  analytically  fireproof  you 
in  this  case.”  She  wouldn’t  settle 
for  anything  less.  She  retires  proud, 
standing  tall  on  a  legacy  of  important 
decisions  that  will  guide  the  direction 
of  the  Air  Force  well  into  the  next 
decades.  Just  as  importantly,  she  has 
faithfully  prepared  those  of  us  who 
have  grown  up  under  her  analytic 
influence  to  carry  the  torch  of  analytic 
light  forward. 
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During  the  late  1960s  and  1970s 
the  Air  Force  studies  and  analyses 
branch  enjoyed  a  respected  position 
in  contributing  to  Air  Force  decisions 
by  leaders  who  fully  understood  the 
value  of  analysis.  Following  the  first 
Gulf  War,  in  an  era  of  the  “peace 
dividend,”  the  USAF  de-emphasized 
independent  analytic  support  for 
decision  making  by  splitting  up  its 
analytic  team  and  relegating  what 
remained  to  lower  levels  in  the 
headquarters.  For  the  first  time  in  its 
history,  the  Air  Force’s  senior  analyst 
did  not  directly  report  to  the  Chief  of 
Staff.  For  almost  a  decade,  Air  Force 
studies  and  analyses  remained  in  the 


shadows  on  the  Air  Staff.  Following 
the  1997  Quadrennial  Defense 
Review,  a  major  institutional  review  of 
the  Air  Force  called  Headquarters  Air 
Force  2002  concluded  that  lack  of  a 
strong,  independent  analytic  function 
was  a  major  issue.  Dr.  Henningsen 
took  charge  of  a  reinvigorated  Air 
Force  Studies  &  Analyses  Agency  as 
a  direct  reporting  unit  to  the  USAF 
Vice  Chief  of  Staff,  returning  analysis 
to  a  respected  place  in  the  Air  Staff. 
On  a  daily  basis  Dr.  Henningsen 
worked  with  the  USAF  leadership, 
providing  analytically  supported 


advice  in  many  areas  including 
planning  and  programming  reviews 
and  Air  Force  contingency  issues, 
including  combat  in  Afghanistan,  Iraq, 
Somalia,  Libya,  and  Syria,  as  well  as 
humanitarian  crises  resulting  from 
hurricanes,  tsunamis,  earthquakes, 
and  events  closer  to  home  that  have 
directly  affected  our  airmen.  This 
now  includes  the  deep  sequestration 
cuts  as  well  as  the  Department  of 
Defense’s  critical  battle  in  the  areas  of 
suicide  and  sexual  assault  prevention. 

Over  her  decades  of  service,  Dr. 
Henningsen  has  accumulated 
numerous  civilian  awards  and 


has  held  various  offices  in  MORS. 

Her  path  is  unique,  having  risen 
from  a  MORS  director  to  two  MORS 
vice  president  positions,  to  a  MORS 
Sponsors  representative  at  OSD,  and 
finally  becoming  the  Air  Force  MORS 
Sponsor.  She  has  been  a  MORS 
Fellow  for  almost  two  decades.  She 
has  been  a  constant  supporter  of 
MORS  as  a  vital  component  in  our 
career  field,  which  was  especially 
supportive  during  some  uncertain 
and  lean  funding  years.  Her  legions  of 
MORS  friends  look  forward  to  seeing 
her  active  in  the  society  for  several 
decades  to  come. 

_ 


accolades  to  include  those  for 
meritorious  and  distinguished  service. 
Most  notable  is  the  President’s 
Award  for  Distinguished  Federal 
Civilian  Service  she  received  in  2012. 
However,  she  is  most  proud  of  the 
achievements  of  those  who  have 
worked  for  and  with  her  through  the 
years.  She  revels  in  hearing  their 
stories,  and  enjoys  nothing  more  than 
a  spirited  discussion  on  innovation  in 
the  science  and  art  of  analytics. 


Dr.  Henningsen  will  pass  the  analytic 
torch  and  the  title  of  senior  “Light 
Shedder”  to  Mr.  Kevin  Williams 
who  has  been  her  Deputy  for  the 
past  five  years.  We  congratulate 
Dr.  Henningsen  on  a  distinguished 
career  and  wish  her  a  fantastic  and 
fulfilling  retirement  as  she  plans  new 
adventures  with  her  husband,  Carl, 
who  has  been  at  her  side  “making 
everything  possible.” 


Dr.  Henningsen  is  also  one  of  our 
most  distinguished  MORSians,  and 
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President,  Military  Operations  Research  Society 

president@mors.org 


Workforce  Management  Analysis 


From  Talent  Acquisition,  Through  Force  Shaping  and  Professional 
Growth,  into  Knowledge  Transfer-/!  Focus  on  a  Critical  Skill 


The  fall  season  is  almost 
behind  us  and  as  we 
approach  winter,  we 
look  forward  to  a  set  of 
tremendous  events  to  close 
201 4  and  kick  off  the  new 
year.  First,  I  would  like  to  highlight 
our  third  Industry  and  Institutions 
Showcase  and  Workshop  to  be 
held  on  December  1 1 ,  at  the 
Hyatt,  Crystal  City,  in  Alexandria, 
Virginia.  The  theme  of  this  event 
is  “Workforce  Development.”  This 
is  a  topic  that  goes  along  with  our 
theme  this  year  focusing  on  the 
growth  of  our  analysts  through 
professional  development  and 
continuing  education. 

There  are  multiple  areas  of  workforce 
management  in  which  scientifically 
based  and  analytically  rigorous 
approaches  are  required.  Many 
experts  in  military  operations 
research  work  today  on  the  multiple 
challenges  of  managing  the  size  and 
the  quality  of  the  force  of  the  military 
services  and  other  national  security 
agencies.  The  efforts  required  in  talent 
acquisition  and  recruiting  are  pivotal 
to  ensure  we  get  the  right  people, 
with  the  right  skills,  to  the  right  places, 
and  at  the  right  time.  Once  the 
workforce  skills  are  present  in  the  right 
locations,  and  in  the  right  numbers, 
organizations  need  to  establish 


strategies  to  ensure  that  the  members 
of  their  workforce  maintain  or  grow 
their  skills,  follow  a  reasonable  career 
progression  path,  and  in  the  process, 
that  some  knowledge  transfer  and 
documentation  occurs  from  the 
lessons  learned. 

Workforce  development  is  an 
approach  that  attempts  to 
enhance  organizational  stability 
and  prosperity  by  focusing  on 
people.  These  strategies  go  beyond 
the  comfort  of  the  individuals, 
identification  with  the  organization, 
and  the  integration  of  a  diverse 
force  at  all  levels.  Personnel 
accessions,  flow  points,  growth, 
and  investment  strategies  are  at  the 
core  of  a  strong  workforce. 

It  would  be  of  great  interest  to  open 
a  dialogue  on  how  government  and 
industry  carry  out  these  activities, 
learn  from  each  other  on  the 
successes  and  failures  experienced 
in  past  and  recent  history,  and 
explore  future  strategies.  There  are 
many  professional  assets  in  the 
human  resources  and  human  capital 
management  disciplines  available  to 
provide  in-depth  context  to  rigorous 
analytic  approaches. 

For  our  Industry  Day,  we  are 
reaching  out  to  professionals  in 


u 


the  workforce  management  area  to 
come  and  discuss  this  topic  of  great 
interest  to  both  government  and 
industry  alike.  I  am  looking  forward 
to  seeing  you  at  this  great  event. 

Fellows  Nominations 

Selection  as  a  MORS  Fellow  is 
the  highest  degree  of  recognition 
that  MORS  bestows  on  our 
members.  Elections  are  conducted 
in  concurrence  with  the  December 
Board  of  Directors  meeting.  We  have 
a  group  of  very  deserving  individuals 
nominated  for  this  honor  this  year. 

I  would  like  to  personally  thank  the 
Membership  Committee  and  our 
membership  at  large  for  submitting 
your  nominees. 


4 


PHALANX  -  DECEMBER  2014 


MORS/MAS 


Dr.  Rafael  E.  Matos _ 

President,  Military  Operations  Research  Society 
president@mors.org 


Sponsor  Retirement  and 
New  Sponsor  Welcome 

On  October  6,  2014,  our  US  Air 
Force  Sponsor,  Dr.  Jacqueline 
Henningsen,  Fellow  of  the  Society, 
retired  from  her  distinguished  service 
as  Director,  Studies  and  Analyses, 
Assessments  and  Lessons  Learned, 
Headquarters  US  Air  Force  (HQ 
USAF/A9).  I  invite  you  to  read 
the  special  feature  in  this  edition 


Headquarters  US  Air  Force,  he 
has  been  responsible  for  the 
development  of  Air  Force-wide 
policy  and  guidance  that  ensures 
defendable  studies,  analyses, 
assessments,  and  lessons  learned 
processes  to  support  Air  Force 
leadership  decisions,  enhancing 
current  and  future  warfighting 
capabilities.  Welcome  to  MORS, 
Mr.  Williams  and  Ms.  Hickman.  We 


the  finest  opportunities  for 
professional  collaboration  and 
networking.  Our  signature  event 
is  scheduled  for  June  22-25, 
2015  at  the  DoD  Mark  Center 
in  Alexandria,  Virginia.  Given 
the  tremendous  attendance  this 
year  at  the  82nd  MORSS,  we 
were  able  to  secure  additional 
rooms  at  the  DoD  Mark  Center 
to  open  up  opportunities  for  the 


of  Phalanx  on  the  tremendous 
contributions  Dr.  Henningsen  made 
throughout  her  career  in  the  US 
Air  Force,  as  an  analyst,  and  a 
lifelong  supporter  of  MORS.  We  are 
very  proud  of  her  support  and  her 
dedication  to  our  discipline. 

At  the  same  time  we  welcome 
Dr.  Henningsen’s  successor  and 
our  new  US  Air  Force  Sponsor, 

Mr.  Kevin  E.  Williams  (see  page 
52)  and  the  Air  Force  Sponsor 
Representative,  Ms.  Patricia  A. 
Hickman  Deputy  Director,  Analyses, 
Assessments  and  Development.  Mr. 
Williams  is  a  member  of  the  Senior 
Executive  Service,  and  as  Deputy 
Director,  Studies  and  Analyses, 
Assessments  and  Lessons  Learned, 


are  looking  forward  to  continue  our 
solid  relationship  with  the  US  Air 
Force  in  support  of  your  operations 
research  analysts’  force. 

Other  Upcoming  Events 

Mark  your  calendars  now  to  attend 
our  Education  and  Professional 
Development  (EPD)  Colloquium, 
March  9-10,  2015,  at  the  George 
Mason  University  in  Fairfax, 
Virginia.  Once  again  MORS 
provides  the  EPD  Colloquium  at 
no  charge  to  participants  and 
offers  experiences  that  help  them 
continue  to  develop  professionally. 

Do  not  forget  to  “lock-in”  the 
dates  also  for  the  83rd  MORS 
Symposium,  which  will  offer 


working  and  composite  groups 
in  an  effort  to  offer  additional 
openings  for  presentations.  The 
call  for  abstracts  is  still  open  and 
I  invite  you  to  take  advantage  of 
this  opportunity  to  submit  your 
work  to  be  considered  to  be  part 
of  this  event.  Details  on  abstract 
submission  and  other  program 
specifics  are  available  in  the 
MORS  website  at  www.mors.org. 

At  the  83rd  Symposium  this  coming 
June,  we  will  also  kick  off  our  50th 
year  with  a  range  of  activities  that  I 
am  sure  will  be  of  great  interest,  as 
we  gear  up  to  celebrate  MORS’  50th 
Anniversary  in  June  2016.  I  am  sure 
many  of  us  are  looking  forward  to 
this  major  milestone  for  our  Society.  I 
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As  is  always  the  case,  there 
are  considerable  “goings 
on”  in  military  operations 
research  and  much  of 
the  action  involves  the 
Military  Applications 
Society  (MAS)  of  INFORMS. 

Thanks  to  all  who  participated  in 
the  annual  INFORMS  conference 
in  San  Francisco  and  the  activities 
of  MAS  during  the  conference.  I 
will  highlight  these  in  the  next  issue 
of  Phalanx.  Thanks  to  all  of  you 
who  have  served  in  keeping  MAS 
an  active,  successful  intellectual 
society  over  these  many  years.  I 
give  my  personal  thanks  to  Bill  Fox, 
who  as  the  president  of  MAS  for  the 
past  two  years  kept  a  steady  hand 
through  some  turbulent  times.  Many 
of  the  MAS  activities  were  affected 
by  government  funding  changes  and 
new  travel  rules  and  requirements. 

I  hope  that  most  of  the  restrictions 
are  behind  us  now.  Also,  my  deepest 
thanks  goes  to  Greg  Parlier,  the 
industrious  and  wise  previous  MAS 
president,  who  always  seems  to 
know  exactly  what  to  do  next  and  is 
amazingly  proactive  in  getting  things 
done.  Also,  all  the  members  of  MAS 
who  were  in  San  Francisco  need  to 
thank  George  Mayernik,  who  once 
again  secured  sponsor  donations 
for  our  excellent  reception  at  the 
conference.  Finally,  I  wish  to  thank 
Walt  DeGrange,  who  has  contributed 
greatly  to  the  organization  and 
smooth  running  of  the  Society  as 
secretary/treasurer  for  these  past 
few  years. 

We  recently  held  a  MAS  election  and 
this  will  serve  as  the  announcement 


of  the  new  MAS  officers  and 
Council.  So  in  the  stead  of  those 
mentioned  above,  Colonel  Andy  Flail 
will  now  serve  as  the  vice  president 
and  my  successor  as  president  in 
two  years.  Colonel  Doug  Matty  will 
be  the  secretary/treasurer.  Serving 
on  the  MAS  Council  will  be:  Mahyar 
Amouzegar,  Chad  Long,  George 
Mayernik,  Aaron  Burciaga,  Natalie 
Scala,  Jeff  Eaton,  Amnon  Gonen, 
Chase  Murray,  Walter  DeGrange, 
Major  Jesse  Pietz,  and  Bill  Fox  as 
the  past  president. 

Our  military  operations  research 
community  is  a  complex  network 
of  people,  societies  (like  MAS), 
organizations,  military  units  and 
agencies,  schools,  colleges, 
universities,  journals,  information, 
ideas,  events,  and  conferences.  I 
like  to  think  of  this  network  as  a  kind 
of  a  military-industrial-educational 
operations  research  complex 
(MIEORC).  The  connections  in 
this  vast  multimodal  maze  include 
friendships,  communicating, 
trusting,  influencing,  collaborating, 
educating,  publishing,  and  reading. 
After  considering  how  to  report 
information  to  you  about  this 
wonderful  MIEORC  complex,  I 
realized  that  it  would  always  be 
too  much  for  this  column.  It  is 
impossible  to  fully  describe  all 
that  happens  in  military  OR  in 
a  short  article.  So  in  my  future 
columns  I  will  try  to  do  my  best  to 
find  the  most  important  and  most 
exciting  activities  to  highlight  and 
continually  remind  all  of  us  that 
much  more  information  is  found 
in  the  rest  of  this  journal,  online 


through  the  INFORMS  website, 
or  announced  through  INFORMS 
new  social  media  forum  called 
INFORMS-Connect.  I  must  admit 
that  what  I  love  about  OR  is  both 
its  strong  network  of  connections 
and  its  bold  engagement  with  the 
practical  (and  therefore  complex) 
problems  of  society.  That  is  to 
say,  OR  is  wonderfully  complex 
and  interdisciplinary!  And  since 
the  military  is  also  complex  and 
interdisciplinary,  OR  and  the  military 
just  seem  to  go  together  quite 
naturally.  My  own  experience  and 
knowledge  is  sparse  in  the  services 
outside  the  Army,  so  I  hope  to 
spend  time  during  my  engagement 
with  the  MAS  community  becoming 
more  interdisciplinary  in  military 
OR  by  learning  about  the  roles 
and  elements  of  OR  in  the  Navy, 
Marines,  and  Air  Force.  Likewise, 
most  of  my  OR  problem  solving 
is  the  in  the  form  of  network 
modeling,  so  I  will  seek  to  expand 
my  own  knowledge  and  use  of 
OR  tools  to  other  models  and 
methodologies.  Perhaps  this  is 
a  role  that  MAS  can  play  for  all 
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members  to  expand  their  breadth  in 
OR  and  management  science. 

In  the  military  OR  community,  there 
are  many  compelling  questions  to 
ask  and  seek  to  answer.  One  basic 
question  is:  To  build  a  modern, 
flexible,  innovative  force,  where  and 
how  should  the  military  invest  in 
people  and  where  and  how  should 
the  military  invest  in  robotics  or 
artificial  intelligence  (Al)  over  the 
next  1 5-20  years?  A  well-informed 
answer  would  guide  investment 
in  people  through  accessions, 
retention,  talent  development,  and 
human  capital  investment  and 
suggest  where  the  military  can 
profitably  apply  robotics,  Al,  and 
other  modern  information-based 
technologies  to  support  its  manpower 
to  achieve  a  flexible,  adaptable  force. 

I  have  been  talking  with  colleagues 
Kira  Hutchinson  (TRADOC)  and  Kate 
Coronges  (Army  Research  Office) 
about  this  enterprise. 

Most  believe  that  the  future 
will  be  unlike  any  world 
that  we  have  experienced 
to  date  and  the  rate  of 
change  is  unprecedented. 

Like  the  global 
marketplace,  modern 
warfare  continues  to 
evolve  at  an  accelerating 
pace.  So  OR  analysts 
must  help  the  military 
adapt  and  innovate.  The 
Army  Science  Board  defines 
innovation  as  something  that 
develops  or  changes  a  product, 
system  or  service,  or  process  to 
produce  greater  results  and  return 
value.  That  definition  calls  for 
OR  to  become  innovation’s  close 


partner.  Innovation  is  the  result  of 
changing,  improving,  adapting, 
or  developing  new,  nascent,  or 
extant  processes  or  products  in  an 
effort  to  achieve  a  desired  result 
while  simultaneously  maximizing 
return  on  investment.  Incremental 
innovation  takes  existing  products, 
systems,  services,  or  processes 
and  improves  them;  it  makes 
them  better  and  increases  their 
value.  Radical  innovation  (possibly 
called  a  paradigm  shift)  does  not 
merely  change  an  existing  product, 
system,  service  or  process,  but 
creates  a  completely  new  way 
of  doing  things  that 
creates  both 
intrinsic  and 


extrinsic  value  for  an  individual  or 
organization. 

Once  established,  an  innovative 
culture  enables  the  workforce 
and  its  partners  to  help  frame  and 
solve  difficult  problems.  Again, 
this  is  precisely  what  OR  does  for 
the  military  and  society.  OR’s  best 
practices  provide  an  opportunity  to 
improve  processes  and  systems, 
allowing  for  increased  effectiveness 
and  efficiency  in  both  operating 
and  generating  forces  regardless  of 
changes  to  the  strategic  environment 
and  national  priorities.  Borrowing 
from  industry, 
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top-performing  companies  have  used 
OR  to  fundamentally  change  their 
approach  to  enhance  innovation  over 
the  past  decade  via  transformational 
information  technology. 

Education  is  extremely  important 
for  OR  and  its  role  in  advancing 
innovation.  We  seek  to  educate 
analysts  on  the  importance  of 
precisely  identifying  problems  and 
challenges  and  engaging  appropriate 
disciplines  and  interdisciplines  to  build 
and  identify  solutions.  Although  the 
introduction  of  appropriate  new  OR 
tools  to  support  innovation  initiatives 


is  important,  education  of  analysts 
and  users  of  OR  must  be  available  to 
support  and  complement  these  tools 
and  processes  as  well. 

What  we  do  in  OR  problem  solving, 
research,  and  education  and  in 
the  various  support  activities  of 
MAS  is  important  in  many  ways. 

OR  involvement  as  the  driver  of 
innovation  provides  the  opportunity 
to  improve  processes  and  systems, 
producing  increased  effectiveness 
and  efficiency  in  military  forces 
regardless  of  changes  to  the  strategic 
environment  and  national  priorities. 


I  hope  all  members  of  MAS  and 
readers  of  Phalanx  had  opportunity 
to  contribute  to  this  innovation 
enterprise  in  2014  and  are  preparing 
for  even  more  achievements  in  201 5. 
Happy  OR-ing  in  2015! 

One  quick  announcement:  You  may 
still  have  time  to  submit  an  abstract 
to  the  Canadian  OR  Society  (CORS)- 
INFORMS  Joint  Meeting  in  Montreal, 
Canada  to  be  held  June  14-17, 

2015.  Deadline  for 
abstracts  is  January  16,  2015. 
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Analytical  Workforce:  Shared  Perspectives  between 
Government  and  Industry.” 

The  Showcase/Workshop’s  purpose  is  to  inform  and 
inspire  attendees  on  current  and  upcoming  challenges  in 
workforce  development  and  human  capital  investments. 
These  challenges  focus  on  two  perspectives.  First,  the 


MORS/M  AS 


he  Military  Operations  Research  Society 
(MORS)  Industry  and  Institution  Partnership 
(IIP)  Committee  will  be  conducting  its  third 
annual  Industry  and  Institution  Showcase 
and  Workshop  on  December  1 1  at  the  Hyatt 
Regency  Crystal  City,  Virginia.  The  showcase/ 
workshop  provides  a  forum  for  government,  industry, 
and  academia  to  understand  and  build  relationships 
on  the  issues  facing  the  national  security  analytic 
community  today.  This  year’s  theme  is  “Developing  the 


INDUSTRY  DAY  SHOWCASE  AGENDA 

Thursday,  11  December  2014 

Hyatt  Regency  Hotel  Crystal  City 

0800  -  0900 

Exhibit  and  Demonstration  Set  Up 

0830  -  0900 

Continental  Breakfast 

0900-0910 

WELCOME 

Mr.  Steve  Notarnicola,  MORS  Industry 
and  Institution  Partners  Chair 

Dr.  Rafael  Matos,  MORS  President 

0910-0955 

Keynote  Address 

Brian  E.  Wilkerson,  Managing  Director 
at  Revolution  Advisors,  LLC 

0955-  1005 

Break 

1005-1050 

Keynote  Address 

LTCOL  Raphael  Hernandez,  National 
Director  of  Advertising,  USMC  Recruiting 

Command 

1050-1100 

Break 

1100-1200 

Panel  Discussion 

Trends  in  National  Security  OR: 
Government  Needs,  Industry  Response 

1200-1345 

Lunch 

Industry  Exhibits  and  Demos 

1345-1430 

Featured  Speaker  from  Industry 

1430-  1445 

Break 

1445-1545 

Panel  Discussion 

Industry  Panel  -  Challenge  of  doing 
Business  in  the  Current  Fiscal 

Environment 

1545-1600 

Closing  Remarks 

Mr.  Steve  Notarnicola,  MORS  Industry 
and  Institution  Partners  Chair 

Dr.  Rafael  Matos,  MORS  President 

1600-1730 

MORS-hosted  Network  Event 
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operations  researcher’s  insights  on  improvements  in  the 
corporate  and  government  workforce  and  the  challenges 
of  attracting,  educating,  and  retaining  talented 
personnel  as  long-term,  valuable  human  resources.  The 
second  objective  is  to  better  understand  and  suggest 
ways  to  enhance  the  development  and  training  of 
operations  researchers  to  meet  the  analytic  demands 
of  government  and  industry.  Through  a  series  of 
distinguished  speakers,  panels  of  experts,  discussions, 
exhibits,  demonstrations,  and  informal  interactions,  the 
showcase/workshop  will  provide  government,  industry, 
and  academia  with  an  opportunity  to  learn  from  each 
other  and  share  experiences  in  a  nonattributional 
multifaceted  forum. 


The  National  Director  of  Advertising  for  the  USMC 
Recruiting  Command,  LTCOL  Raphael  Hernandez,  will 
deliver  the  keynote.  Topics  to  be  addressed  during  the 
workshop  portion  of  the  day  will  include  the  merits  of 
professional  certification,  the  future  government,  and 
civilian  analyst  mix  in  the  national  security  operations 
research  community.  Best  practices  for  meeting  the 
demands  of  the  future  workforce  will  be  highlighted. 
Participants  will  enjoy  the  opportunity  to  discuss  their 


Military  Operations  Research  Society 

3rd  Annual  Industry  Showcase 


Developing  the  Analytical  Workforce:  Shared  Perspectives  between  Government  and  Industry 

11  December  2014  •  Hyatt  Crystal  City 

MORS  is  hosting  our  3rd  Annual  Industry  Showcase  11  December 
2014  at  the  Hyatt  Crystal  City.  The  event  is  an  informative  program 
designed  to  address  challenges  in  Workforce  Development  and 
Human  Capital  Investments.  Keynote  speakers  include  Brian  E. 
Wilkerson,  Managing  Director  at  Revolution  Advisors,  LLC  and 
LTCOL  Raphael  Hernandez  National  Director  of  Advertising,  USMC 
Recruiting  Command.  Panel  discussions  will  examine  trends  in 
National  Security  OR  and  the  challenge  of  doing  business  in  the 
current  fiscal  environment.  In  addition,  there  will  be  the  opportunity 
for  attendees  to  interact  with  the  exhibitors  and  participate  in 
demonstrations.  The  Industry  Showcase  is  the  perfect  forum  for 
government,  industry,  and  academics  to  learn  from  each  other  and 
share  experiences. 


To  register  go  to:  www.mors.org 
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own  challenges  and  successes  during  an  informal 
networking  lunchtime  event. 

The  MORS  Industry  and  Institutional  Partners 
have  been  invited  to  provide  demonstrations  of 
tools,  techniques,  and  applications  that  they  have 
developed  in  their  efforts  to  improve  the  quality  of 
their  analytic  work.  All  attendees  will  observe  and  take 
part  in  the  demonstrations. 

The  MORS  Industry  Showcase  and  Workshop  is  open 
to  all  industry,  academic,  and  government  attendees. 
For  more  information  and  to  register,  please  visit  the 
MORS  website  at  www.mors.org.  If  you  are  interested 
in  becoming  a  MORS  Industry  or  Institution  Partner, 
please  contact  Jennifer  Ferat  at  jennifer@mors.org. 


Coming  Soon  in  Military  Operations  Research 


“Rethinking  the  Hierarchy  of  Analytic  Models  and 
Simulations  for  Conflicts” 

Mark  A.  Gallagher,  David  J.  Caswell,  Brian  Hanlon,  and  Justin  M.  Hill 

This  article  documents  the  rationale  for  a  hierarchy  of  analytic  models 
to  align  with  the  purpose  of  analysis,  applicability  to  the  issues,  efficient 
search  of  the  decision  space,  and  cost.  The  authors  propose  adding 
to  the  current  levels  of  “engineering  and  physics,”  “engagement,”  and 
“mission,”  and  “campaign”  two  more  aggregate  levels  of  “defense 
enterprise”  and  “government,  nongovernment,  and  coalition  instruments 
of  power.” 

“Analytics  in  Action  at  the  New  York  City  Police  Department’s 
Counterterrorism  Bureau” 

E.S.  Levine  and  J.S.  Tisch 

The  New  York  City  Police  Department’s  Lower  Manhattan  Security 
Coordination  Center  integrates  data  from  a  variety  of  sources  and 
records.  The  amount  of  information  received  represents  a  “big  data” 
challenge.  The  authors  describe  the  analytical  processes  that  the  Police 
Department  uses  to  manage  the  incoming  data  and  provide  information 
to  officers  in  the  field,  including  alerting,  categorical  data  analysis,  and 
pattern  recognition. 


“Applying  Hughes’s  Salvo  Equations  to  Engagements 
between  U-Boats  and  Convoy  Escorts” 

Brian  McCue 

In  the  period  1941-1942,  German  U-boats  successfully  engaged  North 
Atlantic  convoys  in  88  encounters,  rederived  from  summary  tables 
via  simulated  annealing,  as  detailed  in  a  recent  Phalanx  article.  These 
battles’  great  number,  their  uniformity  in  technology  and  situation,  and 
their  diversity  of  numbers  of  combatants  and  merchant  vessel  sinkings, 
commend  them  as  a  case  study  for  the  advancement  of  military 
operations  research. 

“Optimization  of  the  Future  Soldier  System:  An  Integrative 
Approach  Based  on  Agent-Based  Simulation  and  Response 
Surface  Method” 

Ingu  Lee,  Youngjung  Geum,  Sung-Pil  Hong,  Subin  Lee,  and  Eui-Jung  Choe 

This  article  explores  the  optimization  of  future  soldier  systems  using  an 
integrative  approach  of  RSM  and  agent-based  simulation.  The  authors 
introduce  decision  and  performance  variables  and  constraints  based  on  a 
literature  review  and  practical  considerations.  They  estimate  the  response 
surface  based  on  an  agent-based  simulation  experiment,  as  well  as  the 
optimal  combination  of  the  soldiers’  equipment. 
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A  Commitment  to 


Life  Long  Learning 

83rd  Symposium  Planning  Update 


Ronda  J.  Syring,  Johns  Hopkins  University  Applied  Research  Laboratory,  ronda.syring@jhuapl.edu 


Our  commitment  to  the  Society’s  mission  — 
“enhance  the  quality  of  analysis  to  address 
real-world  national  security  interests  through 
the  advancement  of  the  OR  profession”  — 
is  as  strong  as  ever.  A  key  contribution  to 
achieving  this  mission  is  through  the  promotion 
of  information  exchange,  as  well  as  through  providing 
collaboration  and  professional  development  opportunities. 
This  promotion  facilitates  the  discussion  of  the  quantitative 
aspects  of  national  defense  and  national  security  in  a 
manner  not  available  under  the  official  channels  and 
processes  of  the  national  defense  community.  The 
principal  venue  for  such  discussions  is  the  annual  MORS 
Symposium.  Planning  is  well  underway  for  the  83rd 
MORSS,  which  will  be  held  at  the  DoD  Mark  Center  in 
Alexandria,  Virginia,  from  June  22-25,  2015. 

Top  DoD  officials  recognize  and  stress  the  importance 
of  maintaining  technical  competence  and  professional 
development  in  order  to  provide  sound  analyses,  insight, 
and  recommendations  to  decision  maker.  With  that  in  mind, 
the  theme  for  the  83rd  Symposium  is  “National  Security 
Analysts:  Growth  through  Professional  Development  and 
Continuing  Education.”  The  goal  is  to  provide  opportunities 
to  develop  or  maintain  analytic  skills  and  knowledge  base 
at  all  levels.  The  next  several  editions  of  Phalanx  will  provide 
details  about  the  plenary  session,  composite  and  working 
groups,  special  sessions,  tutorials,  continuing  education 
unit  short  courses,  demonstrations,  posters,  and  other 
information  specific  to  the  83rd  Symposium. 

Many  volunteers,  in  addition  to  the  outstanding  MORS 
staff,  are  working  together  to  organize  the  83rd  Symposium 
program.  The  symposium  planning  staff  is  listed  below: 

•  Symposium  Chair:  Ronda  J.  Syring 
•  Symposium  Deputy  Chair:  Sheilah  Simberg 
•  Symposium  Advisor:  Rochelle  Anderson 
•  CG/WG  Coordinator:  Jim  Treharne 
•  Deputy  CG/WG  Coordinator:  Randi  VanNyhuis 


•  Virtual  Symposium  Coordinator:  Robert  Henson 

•  Deputy  for  Special  Sessions:  Steve  Stoddard 

•  Tutorials  Coordinator:  Simon  Georger 

•  Continuing  Education  Coordinator:  Darryl  Ahner 

•  Demos/Posters  Coordinator:  Mike  Ottenberg  and 
Rupert  Seals 

•  Prize  Session  Coordinators:  Joe  Adams  and  Chris 
Linhardt 

•  Rooms  Coordinator:  Jim  McMullin 

•  Communities  of  Practice  Coordinator:  Mike  Ottenberg 

•  Security  Coordinators:  Joe  Adams  and  Rupert  Seals 

The  announcement  and  call  for  presentations  (ACP) 
can  be  found  on  the  MORS  website  at  www.mors.org. 
Please  consider  submitting  an  abstract  on  your  relevant 
analyses— either  completed  or  in  progress.  It  is  your 
personal  contribution  that  will  make  the  83rd  MORS 
Symposium  more  professionally  fulfilling  for  all  participants. 


83rd  SYMPOSIUM 

22  -  25  June  2015  •  Alexandria,  VA 
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Vice  Admiral  (VADM)  James  D.  Syring,  Director,  Missile 
Defense  Agency,  will  be  the  plenary  session  keynote 
speaker  for  the  83rd  Symposium.  The  Missile  Defense 
Agency  (MDA)  is  a  research,  development,  and  acquisition 
agency  within  the  Department  of  Defense.  The  MDA 
workforce  includes  government  civilians,  military  service 
members,  and  contractor  personnel  in  multiple  locations 
across  the  United  States.  The  MDA  is  focused  on  retaining 
and  recruiting  a  dedicated  workforce  interested  in 
supporting  our  national  security.  As  the  agency  develops, 
tests,  and  fields  an  integrated  ballistic  missile  defense 
system  (BMDS),  the  MDA  works  closely  with  the  combatant 
commands  (e.g.,  Pacific  Command,  Northern  Command, 
etc.)  who  will  rely  on  the  system  to  protect  the  United 
States,  our  forward  deployed  forces,  and  our  friends  and 
allies  from  hostile  ballistic  missile  attack.  The  MDA  works 


X 


with  the  combatant  commanders  to  ensure  that  MDA 
develops  a  robust  BMDS  technology  and  development 
program  to  address  the  challenges  of  an  evolving 
threat.  The  MDA  is  also  steadily  increasing  international 
cooperation  by  supporting  mutual  security  interests  in 
missile  defense.  The  MDA  is  committed  to  maximizing  the 
mission  assurance  and  cost  effectiveness  of  management 
and  operations  through  continuous  process  improvement. 
You  can  learn  more  about  VADM  Syring  by  reading  his 
biography  published  in  this  edition  of  Phalanx. 

Additional  assistance  is  always  needed.  If  you  would  like 
to  volunteer  to  assist  with  the  symposium  and  share  in  this 
important  MORS  experience,  please  contact  me  at  ronda. 
syring@jhuapl.edu. 


S.  ^  IS  _F 


83rd  MORS  Symposium 

National  Security  Analyst:  Growth  through 
Professional  Development  and  Continuing  Education 

Dr.  Jim  Treharne,  Center  for  Army  Analysis,  James.T.Treharne.civ@mail.mil 


The  Department  of  Defense  and  our  nation  stand  at  a 
critical  crossroads.  The  challenges  and  opportunities 
facing  our  senior  leaders  are  unparalleled  in  recent 
times.  On  September  30,  2014,  at  the  Council  on  Foreign 
Relations  in  Washington,  DC,  Deputy  Secretary  of  Defense 
Bob  Work  remarked, 


attack,  global  mobility  and  logistics,  forcible  entry, 
command,  control,  communications,  and  intelligence 
forces,  and  the  supporting  security  relationships  and 
legal  agreements  that  we  make  in  order  to  facilitate  the 
rapid  concentration  of  forces  in  time  and  space  across 
transoceanic  distances. 


Decisions  we’re  making  every  day  are  really  going  to 
shape  the  Department  for  the  next  couple  decades 
and  determine  in  large  part  on  whether  or  not  we  have 
a  future  that  is  defined  more  by  peace  or  more  by 
crisis.  So  it’s  an  exciting  time,  I  have  to  tell  you. 

It  should  be  an  exciting  time  also  for  the  military  operations 
research  community.  There  have  never  been  so  many 
opportunities  for  our  community  to  impact  the  decision¬ 
making  process  of  our  senior  leaders.  DSD  Work  further 
commented  on  his  thoughts  about  global  posture  based 
on  a  recent  Asia-Pacific  trip. 

It’s  the  deliberate  apportionment  and  global  positioning 
of  our  forward  stationed  and  our  forward  deployed 
forces  and  the  development  of  supporting  global 


No  other  group  is  as  well-positioned  as  the  military  operations 
research  community  to  address  not  only  global  posture 
issues  but  also  the  plethora  of  other  complex  and  interrelated 
issues  facing  the  DoD  and  other  federal  agencies. 

The  83rd  MORS  Symposium  (MORSS)  team  is  well 
acquainted  with  these  issues  and  has  begun  detailed 
planning  to  prepare  the  community  to  tackle  these 
complex  challenges.  Our  theme  this  year  is  “National 
Security  Analyst:  Growth  through  Professional 
Development  and  Continuing  Education.”  If  you  want  to 
learn  fast  and  effectively  from  some  of  our  nation’s  top 
military  analysts,  then  join  us  from  June  22-25,  2015, 
for  four  days  of  education,  networking,  mentoring,  and 
camaraderie.  We  are  very  fortunate  to  have  been  able  to 
secure  the  Mark  Center  Building  in  Alexandria,  Virginia,  for 
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the  majority  of  our  conference  activities.  Just  down  the 
road  from  both  the  Pentagon  and  the  Nation’s  Capital, 
this  great  state-of-the  art  facility  is  perfectly  located 

CG/WG 

83rd  MORSS  Group  Structure 

Description 

CG  A 

Homeland  and  International  Operations 

ano  equippea  to  ensure  me  success  ot  me  symposium 

1/VG  1 

Strategic  Operations  National  Security  Analysis 

and  a  splendid  time  for  its  participants. 

WG  2 

Chemical,  Biological,  Radiological,  Nuclear,  and 

Advanced  Explosive  Defense 

The  strength,  vitality,  and  success  of  our  MORS 

§/VG  3 

Infrastructure  Analyses,  Protection,  and  Privacy 

symposia  are  attributable  to  many  factors:  a  great 
professional  staff,  committed  government  sponsors, 

WG  4 

Homeland  Security,  Homeland  Defense,  and  Civil 

Support 

a  highly  supportive  private  industry,  a  cutting-edge 

CGB 

C4ISR  and  Cyber  Operations 

educational  community,  and  most  importantly,  its 

0/VG  5 

Information  and  Cyber  Operations 

many  volunteers  and  presenters.  Our  presenters  and 

j§/VG  6 

Battle  Management  Command  and  Control 

volunteers  are  the  solid  backbone  of  our  world-class 

0/VG  7 

ISR  and  Intelligence 

organization.  We  believe  that  every  volunteer  and 

1/VG  8 

Space  Acquisition,  Testing  and  Operations 

presenter  is  essential  to  the  success  of  the  organization. 

CG  C 

Joint  Warfare 

We  want  you  to  experience  the  great  personal  and 

1/VG  9 

Air  and  Missile  Defense 

professional  joy  of  being  a  MORS  volunteer  and 

0/VG  10 

Joint  Campaign  Analysis 

presenter.  We  already  have  a  great  team  in  place  for  the 

j§/VG  H 

Land  and  Expeditionary  Warfare 

83rd  MORSS.  But  we  still  have  many  excellent  volunteer 

g/VG  12 

Maritime  Operations 

opportunities.  Let  us  know  you  are  willing  and  able. 

fg/VG  13 

Power  projection  and  Strike 

g/VG  14 

Air  Warfare 

This  year  we  will  once  again  have  a  wide  breadth  of 

CG  D 

Resources/Readiness  Training 

0/VG  15 

Casualty  Estimation  and  Force  Health  Protection 

opportunities  to  both  present  and  grow  as  military 

WG  16 

Strategic  Deployment  and  Distribution 

operations  research  professionals.  We  have  seven 

jg/VG  17 

Logistics,  Reliability,  ad  Maintainability 

composite  groups,  33  working  groups,  and  two 

S/VG  18 

Manpower  and  Personnel 

distributed  working  groups  (see  Table  1).  There 

WG  19 

Readiness 

will  also  be  poster  sessions,  special  sessions,  and 

WG  20 

Analytic  Support  to  Training  and  Education 

tutorials.  After  the  main  conference,  we  will  have 

WG  21 

Operational  Energy 

another  virtual  MORS  session.  Stay  tuned  for  more 

CG  E 

Acquisition 

details  about  this  exciting  initiative  that  MORS 

WG  22 

Experimentation 

continues  to  successfully  mature.  Consider  now  what 

WG  23 

Measures  of  Merit 

analysis  you  would  like  to  present  next  summer.  The 

WG  24 

Test  and  Evaluation 

presentation  can  be  classified  or  unclassified.  The 

WG  25 

Analysis  of  Alternatives 

work  can  be  in  progress  or  complete.  There  is  no 

WG  26 

Cost  Analysis 

better  environment  to  present  your  analysis  and  gain 

WG  27 

Decision  Analysis 

valuable  feedback  in  a  great  collegial  environment.  In 

CG  F 

Interdisciplinary  Advances  in  Operations  Research 

turn,  you  will  learn  methodologies  and  techniques  from 

WG  28 

Modeling  and  Simulation 

peers,  subordinates,  and  seasoned  analysts  that  will 

WG  29 

Computational  Advances  in  Operations  Research 

prove  incredibly  valuable  to  you  in  your  career. 

WG  30 

Wargaming 

CG  G 

Hybrid  Warfare 

We  are  looking  forward  to  seeing  you  this  summer 

WG  31 

Operational  Environments 

WG  32 

Special  Operations  and  Irregular  Warfare 

WG  33 

Social  Science  Methods  and  Applications 

DWG  1 

Human  Behavior  and  Performance 

DWG  2 

Unmanned  Systems 

(CG:  composite  group;  WG:  working  group,  DWG:  Distributed  Working  Group) 
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Vice  Admiral 
James  D.  Syring 
Director,  Missile 
Defense  Agency 

Plenary  Session  Keynote  Speaker  for  the  83rd 
MORS  Symposium 

Vice  Adm.  Syring  is  from  Muncie,  Indiana.  A  1985  graduate  of  the  United  States  Naval  Academy  with  a  bachelor  of  science 
degree  in  marine  engineering,  he  received  his  commission  as  an  ensign.  Subsequent  to  commissioning,  he  was  designated 
an  engineering  duty  officer.  In  1992,  Syring  earned  his  master  of  science  degree  in  mechanical  engineering  from  the  Naval 
Postgraduate  School. 

Ashore,  Syring  served  in  numerous  engineering  duty  officer  assignments  including:  ship  superintendent  for  USS  Port  Royal 
(CG  73);  Aegis  test  officer  for  new  construction  DDG  51  class  ships;  combat  systems,  test  and  trials  officer  in  the  DDG  51 
Aegis  Shipbuilding  Program  Office;  Combat  Systems  Baseline  manager  in  the  Aegis  Technical  Division;  and  director  for 
Surface  Combatants,  Office  of  the  Assistant  Secretary  of  the  Navy  (Research,  Development  and  Acquisition). 

Syring  served  as  the  technical  director  for  the  US  Navy’s  DDG  1000  Shipbuilding  Program  and  followed  that  tour  as  the 
DDG  1000  major  program  manager.  Upon  selection  to  flag  rank  in  2010,  Syring  served  as  the  program  executive  officer  for 
Integrated  Warfare  Systems,  responsible  for  acquiring,  developing,  delivering  and  sustaining  integrated  weapons  systems  for 
ships,  submarines,  carriers  and  aircraft  within  the  Fleet  and  Joint  Force.  In  November  2012,  Vice  Admiral  Syring  became  the 
ninth  director  of  the  Missile  Defense  Agency  (MDA),  Office  of  the  Secretary  of  Defense,  Pentagon,  Washington,  DC.  In  this 
capacity,  he  oversees  the  MDA’s  worldwide  mission  to  develop  a  capability  to  defend  deployed  forces,  the  United  States, 
allies,  and  friends  against  ballistic  missile  attacks. 

Syring’s  personal  awards  include  the  Distinguished  Service  medal,  Legion  of  Merit  (two  awards),  the  Meritorious  Service 
medal  (four  awards),  Navy  and  Marine  Corps  Commendation  medal,  and  Navy  and  the  Marine  Corps  Achievement  medal. 
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Analytics  Rising 

Certified  Analytics 

Professionals 


the  DoD 


Col  Scott  Nestler,  US  Army,  CAR  scott.t.nestler.mil@mail.mil 

Maj  Aaron  Burciaga  (USMCR),  CAR  MORS  Director  and  Education  and  Professional  Development  Chair 


The  opportunities  of  a  data- 
driven  world  have  captured 
the  imagination  and  savvy  of 
businesspersons.  Big  data  is 
in  vogue.  It  is  fueling  a  new 
era  of  business  decisions 
inextricably  tied  to  facts.  Analytics 
increasingly  exist  by  mandate  (Kiron, 
2014)  and  offer  promises  from  Vegas 
to  Bethesda— from  predicting  sports 
scores  (Lewis,  2003)  to  sleuthing 
cures  to  cancer  (Helft,  2014). 

What  kind  of  analyst  will  you  be? 

How  will  you  approach  the  analytic 
challenges  of  the  Big  Data  era?  We 
should  seek  to  be,  and  prefer  to 
work  with,  experts  in  the  fields  of 
analytics.  Education  and  experience 
are  the  hallmarks  of  such  experts. 
Certification  can  be  a  useful  tool  and 
an  important  credential  that  is  also 
the  formal  recognition  of  professional 
attunement.  A  credentialed 
workforce  grows  to  be  the  chorus 
of  true  analytics  professionals 
that  supplant  corporate  pitches, 


A  credentialed 
workforce  grows  to 
be  the  chorus  of  true 
analytics  professionals 
that  supplant  corporate 
pitches,  salesmanship, 
and  less-than- 
thoughtful  delivery. 


salesmanship,  and  less-than- 
thoughtful  delivery. 

There  is  a  clear  need  to  identify 
and  retain  the  analytic  services 
of  professionals  who  have  the 
knowledge,  experience,  and  discipline 


to  lead  data  analytics  projects.  The 
Institute  for  Operations  Research  and 
the  Management  Sciences  (INFORMS) 
has  designed  a  Certified  Analytics 
Professional  program  that  aligns 
with  MORS’  disciplined  approach  to 
excellence  in  operations  research.  The 
CAP®  reflects  important  elements  of 
MORS’  analytic  method  and  other  best 
practices.  During  the  82nd  Symposium 
in  June  2014,  MORS  leaned  forward 
to  partner  with  INFORMS  to  provide 
the  Department  of  Defense  (DoD)  and 
its  partners  with  an  opportunity  to 
be  among  early  adopters  by  taking 
the  INFORMS  Certified  Analytics 
Professional  (CAP®)  exam  at  a 
discounted  rate.  Shortly  thereafter,  CIO 
magazine  named  the  CAP®  as  the  “#1 
Big  Data  certification”  (Olavsrud,  2014) 
(Note:  A  total  of  16  MORS  members 
took  the  exam  at  either  the  symposium 
or  the  201 4  EPD  Colloquium.  Based 
on  reported  employers  of  CAP 
certificants,  about  20  work  for  one  of 
the  military  Services.  Given  that  there 
are  about  50  CAPs  in  the  Washington, 


NOTE 

Under  the  theme  of  “Analytics  Rising,  ”  topics  have  been  selected  for  upcoming  issues  and  guest  contributors  have 
been  asked  to  assist  in  developing  the  content  to  ensure  coverage  and  perspective.  As  we  share ,  provoke,  and 
facilitate  information,  conversation,  and  action  regarding  the  professional  development  throughout  our  ranks,  we  will 

enhance  the  quality  of  operations  research  within  our  community. 
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Table  1.  CAP®  Experience  and  Education  Requirement 


Education  Level 

Field  of  Study 

Experience  Required 

Masters,  PhD 

Analytics-related  area 

3  years 

Bachelors 

Analytics-related  area 

5  years 

Bachelors 

Non-related  area 

7  years 

Table  2.  CAP®  Seven  Domains  in  the  Job  Task  Analysis 


Domain 

Weight 

Business  Problem  Framing 

12-18% 

Analytics  Problem  Framing 

14-20% 

Data 

18-26% 

Methodology  (Approach)  Selection 

12-18% 

Model  Building 

13-19% 

Deployment 

7-11% 

Model  Lifecycle  Maintenance 

4-8% 

DC  area,  there  are  likely  others  working 
for  contractors  in  the  defense  industry.) 

Because  Phalanx  is  a  joint  publication 
of  the  Military  Operations  Research 
Society  (MORS)  and  the  INFORMS 
Military  Applications  Society  (MAS), 
we  provide  a  short  overview  of 
INFORMS  CAP®  and  activities  that 
may  be  of  interest  to  members  of  both 
organizations. 

The  Five  Es 

To  understand  the  value  certifications 
such  as  the  CAP®  offer  the  DoD,  we’ll 
discuss  the  five  components  of  the 
CAP®,  commonly  referred  to  as  the 
five  Es:  Education,  Experience,  Exam, 
Effectiveness,  and  Ethics.  Although 
much  of  the  focus  from  candidates  is 
understandably  on  the  exam,  which 
will  be  discussed  in  detail  shortly, 
the  other  components  are  equally 
important  and  will  be  highlighted  first. 

First,  the  eligibility  requirements  for  CAP 
include  a  combination  of  experience 
and  education,  as  shown  in  Table  1 . 

The  term  “analytics-related  area”  for 


field  of  study  is  relatively  broad,  and 
includes,  but  is  not  limited  to,  analytics, 
operations  research,  management 
science,  statistics,  engineering, 
business  (marketing,  finance,  etc.), 
theoretical  or  applied  mathematics, 
information  technology,  computer 
science,  and  decision  science. 
Applicants  are  required  to  provide  a 
copy  of  their  transcripts  to  INFORMS  as 
part  of  the  application  process. 

Additionally,  there  is  a  requirement  for 
the  verification  of  effectiveness  of  soft 
skills  by  a  current  or  former  employer 
or  client.  The  CAP  doesn’t  just  attest 
to  an  analytics  professional’s  technical 
knowledge,  it  also  includes  confirmation 
of  the  ability  to  communicate  and 
be  effective  throughout  the  analytics 
process.  Finally,  certificants  must 
agree  to  a  code  of  ethics,  which  was 
highlighted  in  a  recent  CIO  magazine 
article  (Pratt,  2014). 

The  CAP  is  not  designed  for  new 
analysts  just  out  of  college,  but  rather, 
an  early-to-mid-career  analytics 
practitioner.  The  exam  is  practice- 


based,  which  means  that  many  of 
the  skills  and  much  of  knowledge 
tested  requires  practical,  and  not  just 
classroom-based,  learning.  It  is  3  hours 
long  and  consists  of  100  multiple- 
choice  questions,  based  on  typical 
tasks  performed  and  knowledge 
applied  by  analytics  professionals.  A 
four-function  calculator  is  provided, 
so  obviously  the  calculations  are  not 
too  demanding,  even  if  the  concepts 
tested  are  challenging.  In  general, 
those  who  have  taken  the  exam  (about 
250,  with  a  passing  rate  of  roughly  70 
percent)  describe  it  as  being  “tough, 
but  fair.”  Very  few  examinees  finish  in 
less  than  2  hours,  but  most  are  not 
there  for  the  entire  time  allowed. 

Seven  Domains 

The  tasks  and  knowledge  statements 
are  organized  into  seven  domains 
in  the  Job  Task  Analysis  (JTA),  an 
analytics  body  of  knowledge  outline. 
As  shown  in  Table  2,  the  JTA  covers 
the  entire  analytic  lifecycle,  starting 
with  the  problem,  working  through 
the  data,  and  finishing  with  model 
deployment  and  monitoring,  thus 
addressing  the  challenges  of  real- 
world  problems.  It  tests  all  domains  a 
cogent  analytics  professional  should 
master.  The  weight  range  next  to  each 
domain  helps  to  assess  each  domain, 
and  represents  the  percentage  (and 
number)  of  questions  in  each  area  on 
any  form  of  the  exam. 

The  INFORMS  CAP®  Candidate 
Handbook  (available  online  at  www. 
informs.org/certification,  along 
with  other  information  about  the 
CAP  program)  contains  a  short, 
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24-question  sample  exam  to  give 
those  who  may  be  interested  an 
idea  of  what  to  expect.  Additionally, 
INFORMS  provides  a  study  guide  that 
is  useful  in  preparing  for  the  exam.  It 
can  be  downloaded  online  as  a  PDF, 
or  you  can  order  a  paper  copy  for  the 
cost  of  printing  and  mailing. 

Continued  Education 

Once  initial  muster  is  passed  on 
the  five  Es  and  CAP  certification  is 
earned,  it  is  valid  for  three  years.  To 
maintain  the  certification,  a  total  of 
30  professional  development  units 
(PDUs)  are  required  during  the  3-year 
period.  There  are  a  variety  of  ways 
to  earn  PDUs— for  example,  as  a 
student  in  formal  analytics- 
related  education/training 
programs,  potentially 
including  those  offered 
by  MORS  (a  minimum 
of  8  hours);  self-directed 
learning  (a  maximum  of 
1 0  hours);  creating  new 
analytics  knowledge  or 
content,  such  as  writing  an 
article  for  Phalanx  or  Military 
Operations  Research,  or 
serving  as  faculty  at  learning 
events  and  other  volunteer 
service  (a  maximum  of  10  hours);  and 
working  as  an  analytics  professional  (a 
maximum  15  hours).  As  you  can  see, 
CAP  is  a  well-rounded  program  of 
continuous  professional  development, 
not  just  a  one-time  event. 

Conclusion 

The  era  of  Big  Data  Analytics 
motivates  our  Services  to  seek  better 
decisions  from  better  information. 


Better  information  requires  better 
analytics.  Better  analytics  requires 
better  analytics  professionals. 
Certifications,  such  as  the  INFORMS 
CAP®,  help  identify  best  analytics 
professionals— those  who  possess 
the  education,  experience, 
effectiveness,  and  ethics  to  lead  and 
deliver  the  methods,  models,  tools, 
and  systems  that  should  characterize 
world-class  analytics  in  defense  and 
security  of  our  nation. 

Consider  distinguishing  yourself 
to  stand  a  bit  taller  and  apart  from 
speculators,  salesmanship,  and  less- 
than-thoughtful  delivery.  By  helping 
MORS  raise  the  bar  for  analytic 


excellence,  you  have  the  opportunity 
to  ensure  a  bright  future  for  yourself 
and  the  MORS  community. 
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SAVE  THE  DATE 

Finally,  save  the  date  for  the  2015  EPD  Colloquium, ,  March  9-10  at  George  Mason  University,  Fairfax,  Virginia.  Mr. 
Chuck  Werchado,  SES,  Deputy  Director  OPNAV  N81,  will  give  the  colloquium’s  keynote.  Colloquium  chair  is  Darryl 
Ahner  (EPD  Cochair).  Registration  details  and  other  event  information  will  be  shared  in  the  next  issue  of  the  Phalanx. 
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201 4M0RS  Awards  and  Prizes 


MORS  Awards 


The  six  MORS  awards  are  the  MORS  Lifetime  Achievement  Award,  the  Wayne  R 
Hughes  Junior  Analyst  Award,  the  Clayton  J.  Thomas  Award,  the  Vance  R.  Wanner 
Memorial  Award,  the  John  K.  Walker,  Jr.  Award,  and  the  MOR  Journal  Award. 


The  MORS  Lifetime  Achievement  Award  recognizes  those  who  are  approaching 
the  end  of  a  professional  career  that  includes  or  incorporates  a  lifetime  involvement 
with  and  outstanding  contributions  to  the  analytical  community  and  MORS. 


The  Wayne  P.  Hughes  Junior  Analyst  Award  recognizes  a  young  operations 
research  analyst  who  has  demonstrated  excellence  in  national  security 
operations  research. 


The  Clayton  J.  Thomas  Award  recognizes  an  individual  who  has,  through  his  or 
her  involvement  in  the  field  of  national  security  operations  research,  exhibited 
sustained  outstanding  individual  performance  and  contributions,  shared 


Each  year,  an  important  focus  of 
MORS  is  its  efforts  to  promote 
and  recognize  exceptionally 
outstanding  professionals  in 
the  national  security  operations 
research  community.  In  addition 
to  valuable  opportunities 
available  to  present  work  at  its 
special  meetings,  symposiums, 
and  colloqui,  the  Society 
sponsors  six  awards  and  two 
prizes  that  span  the  many  ways 
an  individual  can  contribute 


to  the  advancement  of  the 


analytical  profession,  the  national 
security  decision-making 
processes,  and  the  Society. 


knowledge  and  talents  with  others  in  the  field,  and  provided  technically  sound 
options  to  national  security  decision-makers. 


The  Vance  Ft.  Wanner  Memorial  Award  recognizes  a  national  security  operations 
research  professional  who  has  played  a  major  role  in  strengthening  the  profession. 
Winners  have  distinguished  service  over  time  to  the  profession  of  national  security 
operations  research  and  have  demonstrated  sustained  excellence  as  leaders  and 
managers  in  the  conduct  of  national  security  operations  research,  resulting  in 
significant  contributions  to  the  security  of  our  nation. 


The  John  K.  Walker,  Jr.  Award  acknowledges  the  author(s)  of  the  best  technical 
article  published  in  Phalanx,  the  Bulletin  of  Military  Operations  Research.  j 

The  MOR  Journal  Award  recognizes  the  authors)  of  the  best  article  in  Militar 
Operations  Research  (MOR  Journal). 


The  Walker  and  MOR  Journal  Awards  are  based  on  article  submissions.  Those 


wishing  to  be  considered  for  these  awards  must  submit  articles  for  publication 
the  Phalanx  or  Military  Operations  Research  (MOR  Journal).  The  MORS  Lifeti 
Achievement  Award  can  be  presented  at  any  time  during  the  year,  includes  a 
special  article  in  the  Phalanx,  and  requires  initial  personal  contact  with  the  MO 
Executive  Council  to  discuss  the  nominee.  The  Thomas,  Wanner,  and  Hughes 
Awards  focus  on  exceptional  contributions  to  the  profession  over  an  extended 
period  of  time  and  require  nominations.  MORS  provides  the  nomination 
on  its  website  (http://www.mors.org/recognize_excellence/default.aspx). ' 

website  for  nomination  packets.  The  deadline  for  submissior 


To  be  eligible  for  recognition, 
the  Society  provides  two 
approaches.  One  involves 
documenting  your  work  and 
submitting  it  for  presentation  at 
the  annual  MORS  Symposium 
or  publication  in  the  Society’s 
periodicals  (Phalanx  and 
Military  Operations  Research 
journal).  Alternatively,  MORS 
provides  those  who  wish  to 
recognize  the  achievements 
of  their  mentors,  peers,  or 
subordinates  an  avenue  to 
recognize  contributions  through 
the  nomination  process. 


MORS  Awards  provide  avenues 
for  the  recognition  of  young 
analysts  who  have  impacted 
their  organization  positively, 
and  also  for  spotlighting 
distinguished  stalwarts  of 
national  security  analysis 
over  years  of  dedicated 
service.  MORS  prizes  address 
research  projects  by  groups 
and  individuals  as  presented 
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in  our  professional  publications  and  at  the  Society’s 
annual  symposium.  Previous  winners  of  MORS  awards 
and  prizes  can  be  found  on  the  MORS  website  at  www. 
mors.org/recognize_excellence/default.aspx.  In  addition, 
each  year’s  winners  are  listed  in  the  September  edition  of 
Phalanx. 

The  executive  committee,  the  directors,  and  the  members 
of  MORS  encourage  you  to  consider  taking  the  opportunity 
to  recognize  those  who  have  gone  the  extra  mile  for  our 
professional  community  and  the  security  of  our  nation 


through  their  exceptional  national  security  analytical 
contributions.  For  more  information  regarding  MORS  prize 
or  awards,  please  visit  the  MORS  website  (www.mors. 
org).  Click  on  the  link  labeled  “Recognize  Excellence”  and 
follow  to  the  prize/award  sublink  of  your  choice.  If  you  have 
additional  MORS  prize  or  awards  questions,  please  contact 
Awards  Committee  Chairman  Dr.  Arch  Turner  (arch.turn@ 
gmail.com)  for  awards  and  Prize  Committee  Chairman  Joe 
Adams  (jadams@ida.org)  for  prizes.  You  can  also  contact 
Liz  Marriott  (liz.marriott@mors.org)  at  the  MORS  Office  at 
(703)  933-9070  for  further  information  and  links. 


MORS  Prizes 

The  Society  also  annually  presents  two  highly  coveted 
prizes  for  work  presented  in  conjunction  with  the  MORS 
annual  symposium:  the  Rist  Prize  and  the  Barchi  Prize. 

The  Rist  Prize,  named  after  David  Rist  (an  early 
director)  and  first  presented  in  1965,  recognizes 
outstanding  analytical  studies  leading  to  important 
implemented  applications.  The  award’s  criteria 
include  originality  and  ingenuity,  as  well  as  the 
importance  of  the  problem  and  the  impact  of 
the  solution.  Nominations  are  solicited  from  US 
Government  agencies  or  contractors  and  must 
include  a  three-page  abstract  and  a  letter  from  a 
senior  (flag,  Senior  Executive  Service,  or  private 
sector  equivalent)  government  or  industry  study 
sponsor  documenting  the  value  of  the  implemented 
solution.  During  the  83rd  MORS  Symposium,  a  panel 
of  MORS  Fellows  and  Sponsors  will  judge  finalists. 

A  cash  award  is  given  to  the  first  and  second  prize 
winners.  Nominations  are  due  February  6,  2015. 

The  first  place  winner  will  also  give  an  encore 
presentation  at  the  83rd  MORSS.  Nomination 
packets  can  be  found  on  the  MORS  David  Rist  Prize 
website  (http://www.mors.org/recognize_excellence/ 
david_rist_prize.aspx). 

The  Richard  H.  Barchi  Prize,  named  in  honor  of  CDR 
Richard  H.  Barchi,  USN,  a  former  director,  and  first 
presented  in  1983,  is  awarded  to  recognize  the  best 
paper  presented  at  the  last  MORS  Symposium.  It  rewards 
methodological  advances  as  well  as  groundbreaking 
applications.  The  82nd  Symposium  Working  Group, 
Composite  Group,  and  Special  Session  chairs  nominated 
presentations  for  the  2015  award.  Nominees  have  been 
notified  and  asked  to  submit  a  full  paper  by  the  January 
6,  2015  deadline.  A  panel  comprised  of  MORS  Board 
Members  and  outside  experts  will  judge  the  papers. 


The  winners)  will  be  announced  at  the  83rd  MORS 
Symposium  and  will  give  an  encore  presentation  of  the 
work  during  the  Symposium. 

Other  MORS  Opportunities  for  Recognition 

MORS  provides  a  major  commitment  to  its  future— 
the  young  analyst.  Along  with  other  initiatives,  MORS 
sponsors  special  awards  and  competitions  in  this  area. 

A  prized  feature  of  the  MORS  Education  and 
Professional  Development  (EPD)  Colloquium,  the 
Richard  E.  Rosenthal  Student  Competition  uses  a 
“quick  reaction  analysis”  approach.  Students  at  the 
colloquium  are  randomly  teamed  up  to  work  on  an  OR 
problem  together.  They  are  provided  a  problem  set  and 
have  five  hours  to  resolve  the  challenges  in  that  set. 

On  the  last  day  of  the  Colloquium,  each  team  presents 
the  results  of  their  efforts  to  a  judging  panel  in  an  open 
forum.  The  winning  team  receives  a  special  recognition 
by  the  Society  and  each  member  of  the  winning  team 
receives  an  award. 

The  Tisdale  Graduate  Research  Prize  recognizes 
the  outstanding  performance  of  an  operations 
research  student  in  the  graduating  class  at  the  Naval 
Postgraduate  School  (NPS).  The  OR  faculty  at  NPS 
establish  the  specific  standards  for  the  prize  and  select 
the  winner  for  each  class. 

The  Dr.  James  T.  Moore  Graduate  Research  Prize 
provides  a  graduating  OR  student  at  the  Air  Force 
Institute  of  Technology  (AFIT)  a  similar  opportunity 
for  recognizing  his  or  her  outstanding  work.  The 
operational  sciences  faculty  at  AFIT  establishes  the 
specific  standards  for  the  prize  and  selects  the  winner 
for  each  class. 
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Join  the  World  of 
Certified  Analytics 
Professionals  (CAP®) 

Top  5  Reason 


Drms 


Certification  demonstrates  commitment  to  a  profession.  Earning  a  CAP®  credential  shows  your  peers, 
supervisors,  clients,  and  the  general  public  your  commitment  to  analytics  and  your  ability  to  perform 
to  accepted  standards. 

Certification  enhances  the  profession.  CAP®  exists  to  grow,  promote,  and  develop  analytics  professionals 
and  ensures  the  public  that  analytics  is  a  true,  time-honored  profession. 


Certification  reflects  personal  achievement.  Earning  your  CAP®  shows  the  world  you  will  not  stop  learning 
and  will  continuously  improve  your  performance  in  analytics. 


Certification  establishes  professional  credentials.  CAP®  recognizes  an  individual's  accomplishment  and 
stands  out  on  an  individual's  resume  or  a  company's  prospectus. 


Certification  improves  career  opportunities  and  advancement.  CAP®  is  a  clear  identifier  of  those  who  seek 
knowledge  of  changes  in  the  work,  technology,  business  practices,  and  innovation. 


Studies  show  that  those  who  hold  a  professional  certification  like 
CAP®  earn  more  over  their  career  than  peers  who  are  not  certified! 


Join  the  Analytics  Revolution. 


Email  us  at  certification@informs.org  with  any  questions  you  may  have. 
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Evidence-Based 

Decision  Making 

Techniques  for  Adding  Rigor  to  Decision  Support  Processes  in  Complex 
Government  and  Industrial  Organizations 

Dr.  Chris  Hase,  Dr.  Rafael  Matos,  and  Mr.  Don  Styer,  Whiitney,  Bradley  and  Brown  Inc.,  chiase@wbbinc. 
com,  rmatos@wbbinc.com,  dstyer@wbbinc.com 


Government  and  industry 
organizations  today  are 
under  increased  pressure 
to  respond  to  rapidly 
changing  conditions. 
Managers  must  make 
complex,  expensive  decisions,  riddled 
with  risk  and  uncertainty.  Evidence- 
based  decision  making  (EbDM) 
provides  results  where  organizations 
need  to  make  better-informed 


rigor  to  the  decision  support  process 
through  an  emphasis  on  refining 
the  objective,  finding  evidence, 
analysis,  visualization,  and  a  taking 
action  framework.  From  heuristics 
and  optimization  to  simulation  and 
predictive  models,  computer-based 
techniques  provided  traceable, 
repeatable  methodologies  that 
assist  organizations  in  decision 
support.  Our  approach  to  decision- 


decisions,  faster.  Through  the  use  of 
collaboration,  mathematical,  and 
organizational  behavior  tools, 

EbDM  combines  several  ^^4 

technologies  and 

disciplines 


making  support,  EbDM, 
provides  empirical  and 
parametric  evidence 
showing  how 


that  add 


modeling  and  simulation  can  provide 
faster,  more  accurate  reporting, 
improved  decision  making,  improved 
customer  service,  and  reduced  costs. 

The  Art  of  Decision  Making 

“Decision  making  is  a  process 
of  choosing  among  two  or  more 
alternative  courses  of  action  for  the 
purpose  of  attaining  one  or  more 
goals”  (Turban  et  al.,  2011,  41).  In 
Mintzberg’s  (1980)  foundational 
research  on  managerial  work, 
decision  making  was  one  of  the 
top  10  responsibilities  of  managers 
in  the  daily  performance  of  their 
work.  Making  decisions  is  part  of 
every  phase  of  an  operation,  from 
■  organizing,  planning,  executing, 


and  controlling,  to  closing  or 
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Figure  1.  Decision-making  model. 


decision  making  is  synonymous  with 
the  entire  management  process. 

Once  thought  of  as  an  art  acquired 
through  years  of  experience  and 
using  one’s  intuition,  decision  making 
in  organizations  today  is  far  more 
complex,  requiring  institutional 
processes  to  be  able  to  track, 
replicate,  and  defend  the  who, 
what,  where,  and  why  decisions 
were  made  to  stakeholders  and 
regulators  alike. 

Simon  identified  four  phases  of  the 
decision-making  process;  intelligence, 
design,  choice,  and  implementation 
(1977).  Figure  1  (Turban  etal.,  2011, 
46)  provides  a  representation  of 
those  decision-making  phases.  The 
decision-making  process  begins 
with  the  intelligence  phase.  This 
phase  examines  the  organizational 
objectives  surrounding  the  decision, 
initiates  problem  identification, 
ownership,  and  classification.  A 
clearly  defined  problem  statement  is 
an  output  of  the  intelligence  phase. 
The  design  phase  is  characterized 
by  formulating  a  model  that  captures 


qualitative  and 
quantitative 
analysis  leading 
to  a  proposed 
solution.  The  final 
phase  includes 
implementation 
of  the  solution.  If 
implementation 
is  successful, 
the  organization 
moves  forward  on 
to  other  issues. 

If  implementation 
is  not  successful, 
the  decision¬ 
making  process 
is  returned  to  an 
earlier  phase  to  repeat  the  process. 

Turban  (2011)  identified  three 
conditions  under  which  conditions 
are  made.  These  include  decision 
making  under  conditions  of  certainty, 
uncertainty,  and  risk.  “In  decision 
making  under  certainty,  it  is  assumed 
that  complete  knowledge  is  available 
so  that  the  decision 
maker  knows 


course  of  action  will  be  (as  in  a 
deterministic  environment)”  (Turban 
et  al.,  201 1 ,  148).  In  this  environment, 
modeling  and  simulation  have  a 
limited  role  since  the  decision  maker 
has  all  the  information  he  or  she 
needs.  There  are  some  conditions 
that  are  rule  based  and  automated 
decision  systems  can  be  employed. 
The  second  environment  in  which 
decisions  are  made  are  under 
conditions  of  uncertainty.  There 


exactly  what  the 
outcome  of 


elements  of  the  problem  and  its 
relationship  to  attributes  in  the 
system  from  which  it  operates 
The  design  phase  concludes  with 
potential  alternatives  that 
meet  the  criteria  of  solving 
the  problem.  The  choice 
phase  includes  examining 
the  alternatives  through 


are  several  outcomes  for 
each  choice  and 

there  is  usually 
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insufficient  information  for  the 
decision  maker.  Modeling  and 
simulations  are  key  tools  that  can  be 
used  to  equip  decision  makers  with 
the  information  they  seek.  Finally 
there,  is  decision  making  under  risk, 
wherein  “the  decision  maker  must 
consider  several  possible  outcomes 
for  each  alternative,  each  with  a  given 
probability  of  occurrence”  (Turban  et 
al.,  201 1 , 149).  This  environment  is 
also  ripe  for  analytic  tools  that  include 
modeling  and  simulations. 

Framework  for  Business 
Intelligence 

The  concept  of  business  intelligence 
(Bl)  has  gained  acceptance  as  an 
information  system  that  contains  all  of 
the  data  an  executive  needs.  Indeed, 
Bl  is  linked  to  decision  making  but  a 
Bl  system  is  not  a  decision  system 
in  and  of  itself.  Bl  is  relevant  in 
decision  support  since  it  is  viewed 
as  an  overarching  term  that  includes 
architectures,  tools,  databases, 
applications,  as  well  as  methodologies 
(Turban  et  al.,  2008).  Bl  is  based  on 
the  concept  of  transforming  data  into 
information  from  which  decisions  are 
made  and  actions  taken.  This  is  done 
through  interactive  access  to  data 
and  real-time  data  manipulation.  Bl 
contains  four  major  components 
as  part  of  its  architecture:  a  data 
warehouse,  analytical  tools,  a 
performance  management  system 
for  analyzing  performance,  and  a 
user  interface. 

According  to  Thompson  (2004), 

Bl  is  most  commonly  seen  being 
used  in  general  reporting,  sales  and 
marketing  analysis,  planning  and 
forecasting,  financial  consolidation, 
budgeting,  profitability  analysis,  and 
statutory  reporting. 

The  main  benefits  of  Bl  are  to  provide 
accurate  information  when  needed. 
Thompson  (2004)  reported  four  key 
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benefits  of  Bl  systems: 

•  Faster,  more  accurate  reporting 

•  Improved  decision  making 

•  Improved  customer  service 

•  Increased  revenue 

Traditional  Bl  applications  are,  too 
often,  large  monolithic  infrastructures 
that  are  inflexible  and  reliant  on  an 
information  technology  department. 
These  systems  often  answer  only 
predefined  questions,  denying  the 
user  the  ability  to  satisfy  their  curiosity 
and  drill  down  or  look  across  the 
data  in  order  to  answer  questions. 

The  focus  is  too  often  on  data  alone, 
and  not  on  how  the  data  relates 
to  the  vision,  mission,  strategy, 
operational  readiness  requirements, 
and  current  decision  processes.  As 
processes  evolve,  these  systems  do 
not  offer  users  insight  to  the  data  in 
a  manner  that  supports  their  evolved 
responsibilities  or  the  revised  metrics. 
If  the  processes  change,  if  the 
decisions  being  supported  change, 
or  if  the  answers  suggest  additional 
questions  to  the  user,  the  business 
intelligence  capability  has  typically 
not  had  the  ability  to  quickly  adapt. 
The  inability  of  the  analyst  to  explore 
and  ask  additional  questions  of  the 
data  leads  to  frustration  and  does 
not  effectively  support  a  dynamic 
decision-making  process. 

Decision  Support  Systems 

In  assisting  the  decision  maker, 
decision  support  systems  (DSS)  “were 
meant  to  be  adjuncts  . . .  extending 
their  (the  decision  makers’  capabilities 
but  not  replacing  their  judgment.” 
(Turban  et  al.,  201 1 , 75).  Scott-Morton 
described  the  major  concepts  of  a 
DSS  in  the  early  1970s  by  describing 
them  as  “interactive  computer-based 
systems,  which  help  decision  makers 
utilize  data  and  models  to  solve 
unstructured  problems”  (Gorry  and 
Scott-Morton,  1971 , 55).  Yet  others 


provided  many  other  definitions  of 
a  DSS,  leading  to  the  conclusion 
that  there  is  no  universally  accepted 
definition  of  a  decision-support 
system  (Alter,  1980;  Bonczek  et  al., 
1980;  Keen,  1980;  and  Little,  1970). 
However,  there  is  general  consensus 
on  key  characteristics  that  can  be 
found  in  a  DSS,  as  shown  in  Figure  2 
(Turban  et  al.,  201 1 ,  77).  Power  (2002) 
proposed  six  classification  schemes 
for  DSS  that  have  since  been  adopted 
by  the  Association  for  Information 
Systems  Special  Interest  Group  for 
Decision  Support,  Knowledge  and 
Data  Management  Systems  (AIS 
SIGDSS): 

•  Communications  driven 

•  Data  driven 

•  Document  driven 

•  Knowledge  driven 

•  Model  driven 

•  Compound  system  (integrates  two 
or  more  DSS  groups) 

Figure  2  shows  key  characteristics 
that  not  only  comprise  DSS  but  Bl 
systems  as  well.  This  intersection  of 
DSS  and  Bl  systems  lends  itself  to 
a  set  of  tools  and  techniques  that 
define  business  analytics.  It  is  in  this 
arena  that  computer  modeling  and 
simulation  can  yield  the  greatest 
benefits.  These  benefits  include 
(Turban  et  al.,  201 1 , 45): 

•  Manipulating  a  model  (changing 
decision  variables  or  the 
environment)  is  much  easier 
than  manipulating  a  real  system. 
Experimentation  is  easier  and 
does  not  interfere  with  the 
organization’s  daily  operations. 

•  Models  enable  the  compression  of 
time.  Years  of  operations  can  be 
simulated  in  minutes  or  seconds 
of  computer  time. 

•  The  cost  of  modeling  analysis  is 
much  lower  than  the  cost  of  a 
similar  experiment  conducted  on 
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a  real  system. 

•  The  cost  of  making  mistakes  during 
a  trial-and-error  experiment  is  much 
lower  when  models  are  used. 

•  The  business  environment 
involves  considerable  uncertainty. 
With  modeling,  a  manager  can 
estimate  the  risks  resulting  from 
specific  actions. 

•  Mathematical  models  enable 
the  analysis  of  a  very  large, 
sometimes  infinite,  number  of 
possible  solutions.  Even  in  simple 
problems,  managers  often  have  a 
large  number  of  alternatives  from 
which  to  choose. 

•  Models  enhance  and  reinforce 
learning  and  training. 

•  Models  and  solution  methods  are 
readily  available  on  the  Web. 

Evidence-Based  Decision  Making 

Today,  leaders  and  key  personnel 
need  to  be  empowered  to  explore 
and  discover  insights  from  the  data, 
solve  problems,  and  ultimately  make 
informed  decisions  in  a  dynamic 
environment.  DSS  and  Bl  systems 
provide  an  excellent  foundation  for 
constructing  a  framework  that  is 
traceable,  repeatable,  and  defendable, 
yet  flexible  enough  to  adapt  to 
changing  customer  needs.  Building  on 
the  methodologies  and  technologies 
of  DSS  and  Bl  systems,  we  designed 
EbDM,  a  scientific-based  approach 
and  tool  set  designed  to  provide  our 
customer’s  needs.  There  are  five 
elements  of  EbDM  that  are  supported 
by  business  discovery  applications 
(see  Figure  3)  that  are  repeated 
through  a  series  of  sprints  (see  Figure 
4)  until  decision  makers  are  satisfied 
that  their  objectives  have  been  met. 

EbDM  Element  1:  Refine 
Objective 

The  first  element,  refine  the  objective, 
begins  with  an  understanding  of  the 
objectives  of  the  decision-making 
process,  the  strategic  context  within 


how  it  fits  in  the  organization 
and  the  desired  end 
result.  It  is  essential  to  link 
organizational  data  to  the 
most  important  drivers  of 
value  and  performance. 
Decision  makers  must  be 
able  to  describe  they  key 
questions  to  be  resolved, 
from  which  key  performance 
indicators  (KPIs)  are 
developed.  This  in  turn 
informs  the  main  hypothesis, 
relevant  metrics,  and  the 
data  collection  plan. 

EbDM  Element  2:  Find 
Evidence 

The  second  element  in  an 
EbDM  approach  considers 
finding  the  right  evidence 
necessary  to  guide  sound 
decision  making.  Selecting 
only  the  appropriate  data 
critical  to  addressing  the 
key  questions  is  pivotal  to 
finding  the  right  evidence. 

By  understanding  the  KPIs, 
organizations  can  quickly 
sift  through  large  amounts 
of  data  and  focus  only 
on  relevant  information. 
Collecting  and  integrating 
relevant  evidence  is  not  just 
limited  to  quantitative  data 
(numerical  data)  but  also 
qualitative  data  (judgment 
information  that  provides 
context).  Evidence  comes  in 
multiple  forms  that  provide 
context  such  as  numbers, 
sounds,  text,  graphics, 
and  pictures.  Business 
discovery  applications  such 
as  database  management 
systems  (DBMS),  online 
analytical  processing 
(OLAP)  tools,  performance 
management  (BPM/CPM) 
systems,  and  group  support 
systems  (GSS)  are  often 


Figure  2.  Key  characteristics  of  a 
decision  support  system. 


Figure  3.  Five  elements  of 
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Figure  4.  Employing  lean  startup 
methodologies  with  EbDM. 
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used  here. 

EbDM  Element  3:  Analyze 
The  third  element,  analyze,  focuses 
on  transforming  critical  data  into 
actionable  knowledge.  Many 
organizations  are  so  focused  on 
the  collection  and  distribution 
of  data  that  there  is  little  effort 
placed  on  meaningful  analysis.  To 
overcome  these  shortfalls  a  rigorous 
methodology  that  includes  discovery, 
diagnoses,  prescription  and  prediction 
is  needed. 

Discovery.  The  analysis  starts  with 
developing  a  complete  understanding 
of  the  descriptive  nature  of  the  data. 
This  builds  insights  that  identify 
statistical  associations  among  events 
or  observations  and  help  to  confirm 
causal  relationships.  Looking  at  the 
data  from  different  perspectives 
proves  or  disproves  hypotheses 
generated  during  the  framing  and 
evidence  gathering.  The  exploration 
allows  for  the  identification  of  hidden 
trends  and/or  gaps  in  the  data. 
Discovery  is  an  iterative  process  of 
continuous  profiling  (what  it  is,  who 
it  belongs  to,  where  it  is  used)  and 
validating  (identification  and  mitigation 
of  flaws)  the  data. 

Diagnoses.  The  key  analytics 
questions  and  KPIs  provide  the 
foundation  of  the  diagnostics  analysis. 
Through  the  use  of  business  discovery 
applications,  such  as  optimization 
models,  mathematical  programming, 
trend  analysis,  and  forecasting,  a 
higher  degree  of  analytics  can  be 
achieved.  This  enables  the  team  to 
quickly  drill  into  root  causes  and 
identify/implement  appropriate 
business  rules,  algorithms,  and 
mathematical  models. 

Prescription.  Reports  and  queries 
are  performed  against  databases 
to  address  decision  makers’ 


questions  and  produce  prescriptive 
recommendations.  Given  the  growth 
of  data  and  the  shortened  decision 
cycle  time,  KPIs  are  programmed  into 
a  business  discovery  dashboard.  This 
enables  the  analysts  and  decision 
makers  to  rapidly  identify  the  issues, 
refine  their  questions,  and  develop  the 
necessary  information. 

Prediction.  Data  is  transformed 
for  use  in  predictive  models  and 
integrated  into  the  business 
discovery  platform.  The  predictive 
models  are  used  in  trend  analysis 
to  generate  forecasts  with  well- 
characterized  accuracies  about 
the  future  or  diagnoses.  Such 
forecasts  or  diagnoses  can  be 
harnessed  within  procedures  that 
generate  recommendations  to  the 
analyst  on  how  to  react  to  what 
the  data  represents.  The  cycle  of 
data-prediction-action  provides  a 
pervasive  decision  support  capability 
engendering  decision  confidence. 

The  key  to  EbDM  is  the  ability  to 
rapidly  provide  a  pervasive  analytical 
delivery  mechanism  enabling  a 
whole  new  level  of  analysis,  insight, 
and  value  to  existing  data  stores 
with  user  interfaces  that  are  clean, 
simple,  and  straightforward.  Using 
a  business  discovery  platform 
simplifies  the  analysis  using  a  variety 
of  user  driven  interactive  and  intuitive 
presentations.  The  dashboard 
becomes  the  “glue”  to  conducting 
descriptive,  diagnostic,  prescriptive, 
and  predictive  analysis. 

Element  4:  Visualize 
It  is  crucial,  when  analyzing  data, 
to  keep  the  target  audiences  and 
their  specific  needs  in  mind.  EbDM 
is  only  fully  effective  when  the  right 
information  is  delivered  to  the  right 
people  at  the  right  time.  Business 
discovery  tools  include  geographical 
information  systems  (GIS), 


informational  portals,  multidimensional 
presentations,  and  dashboards. 

Throughout  the  previous  steps, 
stakeholders,  analysts,  and  decision 
makers  were  identified  who  interact 
with  the  data.  The  basis  for  the  design 
of  the  interactive  user  interface  comes 
from  the  decision  process  models 
and  use  cases.  This  provides  context 
to  what  will  follow  and  ensures 
that  the  charts,  graphs,  and  tables 
are  focused  squarely  on  meeting  a 
critical  information  need  of  the  target 
audience.  This  avoids  the  trap  of 
focusing  on  “interesting”  rather  than 
“valuable”  information. 

In  traditional  models  that  follow  a 
linear  path  of  analysis,  presentation, 
and  decision  maker  feedback,  time 
is  wasted  between  receiving  decision 
maker  feedback  and  cycling  back 
through  analysis  and  presentation 
to  provide  answers  to  the  decision 
maker’s  previous  questions.  In  a 
visualization  model  that  uses  adaptive 
dashboard  techniques,  the  linear 
model  is  replaced  with  a  circular  model 
where  the  decision  maker  is  part  of  the 
analysis  visualization  cycle  and  instead 
of  waiting  for  feedback  from  the 
decision  maker,  supporting  analysts 
are  more  apt  to  move  directly  into  the 
“take  action”  element  of  EbDM. 

Sometimes,  decisions  must  be  made 
under  conditions  of  risk,  when  there 
are  multiple  outcomes  each  with  its 
own  probability  distribution  function. 
Or,  sometimes  there  is  just  insufficient 
information  to  make  a  decision  and 
the  data  does  not  exist  that  can 
help  inform  the  decision  maker. 

In  situations  like  these,  a  different 
approach  must  be  taken  to  assist  the 
decision  maker.  One  methodology 
designed  for  situations  like  these, 
called  Lean  Startup,  was  developed 
by  Eric  Ries  (2011).  Lean  Startup  was 
initially  developed  for  technology- 
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driven  startup  companies  but  has 
been  adapted  to  industry  in  general 
faced  with  the  need  to  make  decisions 
with  less  than  ideal  information.  Lean 
Startup  aims  to  shorten  product 
development  cycles  by  adopting  a 
combination  of  business-hypothesis- 
driven  experimentation,  iterative 
product  releases,  and  validated 
learning  principles.  Applying  a  Lean 
Startup  methodology  to  EbDM 
produces  a  series  of  repeated 
cycles  or  sprints  that  each  produce 
a  viable  prototype  and  build  on 
lessons  learned  from  earlier  efforts 
(see  Figure  4).  Users,  stakeholders, 
and  decision  makers  work  closely 
to  discover,  validate,  improve,  and 
pivot,  if  necessary,  throughout  each 
iteration.  This  drives  immediate 
value  and  gives  stakeholders  control 
over  the  outcome.  The  intent  is  to 
capture  inputs  early  by  starting  small, 
incorporating  user  collaboration,  and 
then  building  incremental  capability. 
The  process  focuses  on  critical 
decisions,  processes,  required  data, 
and  KPIs. 

EbDM  Element  5:  Take  Action 
Adoption  of  EbDM  enabled  by  a 
business  discovery  application 
provides  widespread  analytical 
capabilities  across  an  organization, 
allowing  it  to  exploit  fleeting 
opportunities  in  a  budget-constrained 
environment.  However,  streamlining 
decision  processes  often  requires 
active  change  management  that 
builds  successful  practices  into  the 
beliefs  and  culture  of  an  organization, 
enabling  faster  and  more  effective 
reactions  to  external  events.  As 
described  in  the  visualization 
element,  effective  use  of  dashboards 
depends  on  using  the  right  business 
discovery  tools  and  incorporating  the 
correct  KPIs  and  analytics  to  deliver 
the  information  decision  makers 
need  to  develop  a  knowledge  base 
sound  enough  to  make  a  traceable, 


repeatable,  defendable  decision. 
Sometimes  this  can  occur  in  one 
event.  Alternatively,  discovery  is 
made  and  a  different  prototype  of  the 
model  must  be  developed  to  address 
different  questions  and  KPIs.  Figure 
4  captures  this  iterative  process  of 
building  on  discovery  through  the 
use  of  repetitive  prototypes,  each 
designed  to  provide  decision  makers 
with  the  knowledge  base  they  require. 
Business  discovery  applications  that 
commonly  use  this  element  include 
multicriteria  decision  making  with 
pairwise  comparisons  and  analytic 
hierarchy  process  (AHP)  techniques. 

Conclusion 

EbDM  enables  government  and 
industry  leaders  to  rapidly  achieve  a 
cross-functional  advanced  analytic 
capability.  This  methodology  is 
scientific  and  aligns  data  collection  to 
strategic  value  drivers,  and  collects 
the  best  available  evidence.  This 
evidence  is  then  used  to  extract 
valuable  knowledge  and  sharing 
analytics  in  a  way  that  allows  all 
users  to  act  on  those  insights.  In 
short,  the  approach 

•  Provides  evidence-based 
business  discovery  that  lets 
users  ask  questions  of  data,  thus 
effectively  gaining  insight  from 
relevant  data. 

•  Installs  rapid  access  to  multiple 
federated  data  sources  to  monitor, 
measure,  and  manage  operational 
performance,  resources, 
requirements,  and  project  status, 
as  well  as  the  relationships  and 
dependencies  among  them. 

•  Provides  the  analytical  tools  that 
support  analysts  and  decision 
makers,  giving  them  the  ability 
to  quickly  discover  and  assess 
shortfalls  in  required  data,  support 
tradeoff  decisions,  and  assess 
risk  in  near  real  time. 

•  Quickly  collaborate  across  the 


organization  by  sharing  content 
and  filtered  data,  annotating 
elements,  sharing  snapshots  of 
their  data  set,  or  sharing  their 
session  and  enabling  guests  to 
actively  make  selections. 
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Learn  more  at  lockheedmartinjobs.com/phalanx 
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Words  from  Yesteryear 


Brian  McEnany,  FS,  bmcenany@cox.net;  and 

Roy  Rice,  FS, Teledyne  Brown  Engineering,  Roy.Rice@tbe.com 

Several  years  ago,  a  group  of  us  pulled  together  a  series  of  sayings  that  were  found  in  various  operations  research 
briefings,  papers,  and  books  and  published  it  in  the  Phalanx.  Most  of  those  quotes  had  been  used  by  OR  analysts  in 
various  presentations  over  the  years.  We  found  that  briefings  tend  to  be  boring  and  a  little  humor  helps  get  things  started 
sometimes.  With  a  little  help  from  the  Fellows  of  the  Society  and  colleagues,  Roy  and  I  offer  the  following  update  to  our 
collections  of  OR  sayings: 


General  Comments  on  Analysis  and 
Research 

These  quotations  were  gathered  from  different  projects 
and  briefings.  You  should  ensure  their  applicability  before 
inserting  them  into  your  briefings.  (Ignore  them  at  your  peril!) 

The  most  important  one  of  all  that  we  must  never  forget  is 
Murphy’s  Law:  If  something  can  go  wrong— it  will. 

•  Who  is  really  an  analyst: 

♦  Sitting  in  a  room  where  others  run  models  no  more 
makes  you  an  analyst  than  sitting  in  your  garage 
makes  you  a  car.  —Dr.  Kirk  Yost 

•  How  one  gains  experience: 

♦  Good  judgment  comes  from  experience  and  a  lot 
of  that  comes  from  bad  judgment. 

—Will  Rogers 

•  Keep  it  simple,  Stupid  (KISS)  is  a  principle  to  which  all 
should  profess  adherence.  It  is  based  on  Ockham’s 
Razor  (1280-1340): 

♦  Multiplicity  ought  not  to  be  posited  without 
necessity. 

•  It  is  quite  at  odds 
with  the 


weight  and  heft  of  some  study  reports  even  as  we  try 
to  explain  the  unexplainable.  Our  efforts,  sometimes, 
can  be  related  to  a  quotation  from  the  Bible: 

♦  Come  let  us  go  down  and  there,  confuse  their 
language  that  they  may  not  understand  one 
another’s  speech. 

—Genesis  1 1 :7  (Tower  of  Babel) 

On  analysis: 

♦  Look  for  the  analogies  and  identify  the  similarities 
and,  most  particularly,  the  differences. 

—Gene  Visco,  FS 

Study  plans  are  not  that  dissimilar  to  military  plans  and 
one  hears  frequently  that  the  military  always  invokes 
von  Molke’s  quotation: 
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♦  No  plan  survives  contact  with  the  enemy. 

—  Field  Marshall  von  Molke 

♦  During  a  Desert  Storm  briefing  in  1992,  General 
Schwarzkopf  announced  his  corollary  to  von 
Molke  principle:  A  plan  is  liken  to  a  director  of  an 
orchestra.  It  follows  the  score  until  some  SOB 
comes  out  of  the  orchestra  pit  and  chases  the 
director  around  the  stage  with  a  bayonet. 

•  As  in  all  projects,  costs  and  benefits  must  be  weighed. 

♦  Nullium  gratuitum  pradium— there  is  no  free  lunch. 
—GEN  Max  Thurman 

Data  and  the  Use  of  Numbers 

We  deal  with  numbers  and  data  as  a  matter  of  course  in  the 
analysis  of  studies,  models,  and  simulations.  Often,  the  mere 
“tweaking”  of  numbers  causes  results  to  vary  widely— and 
not  just  because  of  sensitivity.  Several  of  these  sayings  may 
be  applicable  where  the  acquisition  of  data  and  its  conversion 
into  numbers  is  a  major  part  of  the  conduct  of  a  study. 

•  Quantify,  quantify,  quantify: 

♦  Measure  what  can  be  measured,  and  make 
measureable  what  cannot  be  measured. 

—Galileo 

•  How  much  detail  and  fidelity: 

♦  With  four  parameters  I  could  fit  an  elephant,  with 
five  his  trunk  would  wave. 

—  Poincare  (French  mathematician) 

•  What  does  the  data  tell  us: 

♦  Analysts  torture  data  until  it  confesses. 

—Vince  Roske,  J8  (now  IDA) 

♦  The  one  thing  we  expect  from  the  bean-counters 
is  an  accurate  count  of  the  beans. 

—  Dave  Hardison  (DUSA,  OR) 

♦  If  all  else  fails,  get  the  facts. 

—  LTG  Bill  Richardson,  former  USA  DCSOPS 

♦  Gather  the  facts  first,  dispute  them  later. 

—  Mark  Twain 

•  Importance  of  measurement: 

♦  /  often  say  that  when  you  measure  what  you  are 
speaking  about,  and  express  it  in  numbers,  you 
know  something  about  it,  but  if  you  cannot  measure 
it  when  you  cannot  express  it  in  numbers,  your 
knowledge  is  of  a  meager  and  unsatisfactory  kind.” 
—  Lord  Kelvin,  1883 

•  Precision  of  measurement: 

It  is  the  mark  of  an  educated  mind  to  rest  satisfied 
with  the  degree  of  precision  which  the  nature  of  the 
subject  admits  and  not  to  seek  exactness  where  only 
an  approximation  is  possible. 

—Aristotle 


It’s  better  to  be  approximately  right  than  precisely  wrong. 
—Anonymous 

•  Believability: 

♦  There  are  three  kinds  of  lies:  lies,  damned  lies  and 
statistics  —generally  credited  to  Benjamin  Disreali 

•  Predictions: 

♦  Every  missile  on  a  drawing  board  has  a  Pk  -  1.0. 

—  Dr.  Roy  Rice,  FS 

♦  The  purpose  of  mathematical  programming 
is  insight,  not  numbers.  ”  —Arthur  Geoffrion, 
Interfaces  Vol  7,  No  1 ,  1 976. 

Models,  Simulations, 
and  Analysis 

The  use  of  M&S  is  often  confused  with  analysis.  Clayton 
Thomas,  FS,  once  said  that  we  should  work  toward 
having  Big  “A”  and  little  “m&s.”  In  other  words,  spend 
more  time  on  analysis  rather  than  just  running  models  and 
simulations  to  get  results. 

•  The  use  of  models  is  too  frequently  equated  with 
analysis: 

♦  Models  are  for  thinking  with.  —Sir  M.G.  Kendall 

♦  Models  never  perform  analysis.  Analysts  do 
analysis,  —believed  to  be  from  Judy  Grange  and 
John  Battilega 

•  On  making  models  more  user  friendly: 

♦  If  you  build  a  model  that  even  idiots  can  use,  rest 
assured,  idiots  will  use  it. 

-Dr.  Wilbur  Payne  (DUSA, OR) 

•  Assumptions  and  key  factors: 

♦  To  omit  such  (soft)  variables  is  equivalent  to  saying 
that  they  have  zero  effect— probably  the  only  value 
that  is  known  to  be  wrong.  —Forrester,  1961 

♦  Or  it’s  shorter  version:  To  omit  a  factor  is  to 
assume  that  it  is  zero.  —Dr.  Paul  Davis 

•  Clayton  Thomas  referred  to  John  Tukey  with  his  words 
about  models  and  simulations. 

♦  All  models  are  wrong;  some  are  useful. 

•  On  verification  and  validation: 

♦  There  is  no  such  thing  as  verification  and 
validation  of  models.  Models  are  “corroborated” 
and  “calibrated.  ” 

—  Dr.  Wilbur  Payne 
(DUSA,  OR) 

♦  W&A  is  not  a  destination;  it’s  a  journey. 

—Clayton  Thomas 

♦  Happiness  is  getting  the  same  answer  twice  with  a 
deterministic  model. 

—  Brian  McEnany,  FS 
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•  On  cost  models  and  cost  analysis: 

♦  When  a  technical  analyst  makes  a  presentation, 
everyone  but  the  analyst  believes  the  results. 

When  a  cost  analyst  makes  a  presentation,  no  one 
but  the  analyst  believes  the  results. 

—Anonymous 

The  Profession 

We  analysts  sometimes  have  to  look  hard  to  find  academic 
programs  that  foster  operations  research  as  a  discipline 
nowadays.  Sometimes,  we  look  back  to  find  a  clear 
definition  of  what  the  profession  is  all  about.  OR  definitions 
should  not  be  limited  to  140  characters.  Here  are  a  few  we 
have  gathered  over  time. 

•  Dr.  Phillip  M.  Morse  and  George  W.  Kimball,  two 
founding  OR  analysts  who  date  to  work  undertaken 
during  WWII,  began  their  book  Methods  of  Operations 
Research  with  the  following  words: 

♦  Operations  research  is  a  scientific  method  of 
providing  executive  departments  with  a  quantitative 
basis  for  decisions  regarding  the  operations  under 
their  control,  (p.  1 ) 

•  Prof  Robert  E.  (Gene)  Woolsey,  at  the  Colorado 
School  of  Mines,  stated  at  one  point  a  pretty  distinct 
definition: 

♦  OR  is  the  application  of  logic  and  mathematics 
to  a  real  world  problem  in  such  a  way  that  the 
method  doesn’t  get  in  the  way  of  common  sense. 

•  Organizations: 

♦  We  trained  hard— but  it  seemed  that  every  time 
we  were  beginning  to  form  into  teams,  we  would 
be  reorganized.  I  would  learn  that  later  in  life  we 
tend  to  meet  any  new  situation  by  reorganizing, 
and  a  wonderful  method  it  can  be  for  creating  the 
illusion  of  progress  while  producing  confusion, 
inefficiency  and  demoralization.  —  Petronius 
Arbiter,  a  Roman  General,  noted  in  66  AD 

•  On  the  need  for  military  decision  leaders  to  study 
and  reflect: 

♦  The  nation  that  will  insist  on  drawing  a  broad 
distinction  between  the  fighting  man  and  the 
thinking  man  is  liable  to  find  its  fighting  done 
by  fools  and  thinking  done  by  cowards.  —Sir 
William  Francis  Butler,  19th  Century  British 
soldier  and  author 

•  Russell  Ackoff,  in  his  1962  book,  quoted  Thomas 
Huxley,  who  stated  that 

♦  Science  is,  I  believe,  nothing  but  trained  and 
organized  common  sense,  differing  from  the  latter 
only  as  a  veteran  may  differ  from  a  raw  recruit:  and 
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its  methods  differ  from  those  of  common  sense 
only  so  far  as  the  guardsman  cut  and  thrust  differs 
from  the  manner  in  which  a  savage  wields  his  club. 

•  Paul  Davis  wrote  in  a  recent  Rand  book  about  a  new 
professional  responsibility  for  analysts:  Instead  of 
merely  listing  assumptions,  analysts  should 

♦  Routinely  show  how  results  vary  with  all  key 
assumptions  and  disagreements— the  opposite 
of  focusing  on  a  standardized  case  and  perhaps 
running  a  few  excursions, 

♦  Routinely  assess  options  for  FARness,  showing 
he  value  of  affordable  hedges  even  in  period  of 
austerity  when  hedges  may  seem  like  luxuries, 

♦  Do  the  above  comprehensibly  to  aid  policy  makers 
in  converging  on  decisions  and  actions. 

Problems  and  Their  Definitions 

At  the  heart  of  every  defense  issue  lies  a  real  problem  that 
may  or  may  not  be  understood.  It  is  our  task  as  operations 
research  analysts  to  understand  it,  define  it,  and  determine 
the  best  approach  to  reach  a  solution.  The  task  requires 
thinking  that  is  not  subject  to  Moore’s  Law. 

All  advocates  focus  on  how  important  the  problem  definition 
is  in  any  analytic  study  project.  It  is  the  first  and  most 
important  step.  Both  Roy  Rice,  FS,  and  Andy  Loerch,  FS, 
have  given  presentations  at  past  MORS  symposia  and 
education  and  professional  development  colloquia  designed 
to  focus  attention  on  the  importance  of  problem  definition. 

•  Understanding  the  problem: 

♦  If  you  do  not  know  the  problem,  then  you  can’t 
analyze  or  solve  it.  —Priscilla  Glasow,  FS 

•  Danny  Weil,  a  former  MORS  member,  offered  the 
following  statement  in  2001 : 

♦  Fifty  percent  of  all  problem  solving  is  defining 
the  problem. 

•  Major  General  Dave  Robinson  (former  J8)  and  Vince 
Roske,  FS,  have  put  the  first  step  in  more  common 
terms.  Both  advocate  the  following: 

♦  What  is  the  question?  What  is  the  real  question? 
What  does  the  final  slide  look  like?  What  do  we 
know  already?  Flow  do  we  get  the  missing  data? 

—  MGEN  Dave  Robinson,  Joint  Staff/J8 

•  It  seems  to  be  simple  common  sense,  but  in  the 
process  of  defining  the  problem,  other  elements 
must  be  defined  as  well.  This  was  captured  by  an 
anonymous  Chinese  philosopher  many  years  ago: 

♦  The  beginning  of  wisdom  is  to  call  things  by  their 
right  names. 

•  I  added  at  one  point  that  you  must 
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♦  Understand  the  problem  first,  and  the  method  for 
its  solution  will  more  easily  follow. 

—  Brian  McEnany,  FS 

•  The  inimical  Gene  Woolsey,  Colorado  School  of  Mines, 
offered  his  view  based  on  observation  and  studies  he 
had  conducted: 

♦  A  manager  would  rather  live  with  a  problem  that 
he  cannot  solve  than  accept  a  solution  that  he 
cannot  understand. 

•  My  own  observation  after  dealing  with  defense  issues 
for  many  years  is  that  many  are  cyclic  in  nature. 
Analysts  must  research  past  attempts  to  ensure  that 
their  view  of  the  problem  is  better  than  what  was 
posited  before. 

♦  If  we  cannot  remember  the  past,  we  shall  be 
condemned  to  repeat  it. 

—George  Santayana 

♦  Every  time  history  repeats  itself  the  price  goes  up. 
—Anonymous 

♦  If  we  do  not  change  the  future  we  shall  be 
compelled  to  endure  it— and  that  could  be  worse. 
—Alec  Toffler 

♦  What  is  past  is  prologue  —engraved  over  the 
entrance  to  the  National  Archives;  Shakespeare; 
from  The  Tempest,  Act  II,  Scene  1 ,  lines  253-254 

Project  Management 

The  analytic  process  lays  out  the  process  and,  most 
importantly,  the  analyst  must  spend  time  and  energy  on  its 
first  step.  There  are  metrics  dealing  with  theoretic  timelines 
for  the  conduct  of  an  analytic  project— for  example,  the 
US  Army  Logistics  Management  Center  at  Fort  Lee  (1 989) 
offered  the  following:  “Forty  percent  of  the  time  devoted  to 
problem  definition  and  front  end  analysis,  twenty  percent 
to  crunching  the  numbers,  and  forty  percent  to  examining 
the  answers  and  packaging  the  results.”  Others  advocate 
one-third,  one-third,  one-third  to  those  same  important 
events.  Personally,  I  have  advocated  the  importance 
of  front-end  analysis  as  a  precursor  to  actual  problem 
solution— as  a  means  of  concentrating  effort  on  defining 
the  real  problem  and  selecting  an  appropriate  analytic 
methodology  to  solve  it.  Then  a  second  phase  proceeds  to 
the  actual  solution. 

•  Time  well  spent: 

♦  Knowing  (understanding)  what  the  problem 

is  in  the  first  place  is  half  the  project  time  and 
reporting  the  results  so  that  the  intended  audience 
understands  the  results  and  can  act  upon  them  is 
one  third  of  the  project.  Analysis  is  only  one  sixth 
of  the  problem.  —  Priscilla  Glasow 


•  Although  we  do  not  advocate  exceeding  executive 
authority,  E.B.  Vandiver  related  that  GEN  Max  Thurman 
once  stated: 

♦  While  it  is  true  that  the  meek  shall  inherit  the  earth, 
in  the  meantime,  the  bold  will  run  it. 

—GEN  Max  Thurman 

♦  Max’s  Rule  13  was,  When  in  charge  take  charge, 
and  Rule  14  was,  If  no  one  is  in  charge,  take  charge. 

•  An  early  ORSA  practitioner,  P.M.S.  Blacket,  once  stated 
that,  [Tjhough  the  research  workers  should  not  have 
executive  authority,  they  will  certainly  achieve  more 
success  if  they  act  as  if  they  had  it.  I  mean  by  this  that 
when  an  operational  research  worker  comes  to  some 
conclusion  that  affects  executive  action,  he  should  only 
recommend  to  the  executives  that  the  action  should  be 
taken  if  he  himself  is  convinced  that  he  would  take  the 
action,  were  he  the  executive  authority. 

—  Blackett  provided  this  message  in  a  variety  of 
forms  in  a  number  of  his  essays  after  the  war;  this 
was  one  of  the  cumbersome  ways  he  conveyed 
the  message.  From  “The  Scope  of  Operational 
Research,  1950,”  Operational  Research  Quarterly, 
Vol  1,  No  1,  Mar.  1950. 

Reporting  Results 

Reporting  results  is  a  vital  part  of  any  analytic  effort. 

You  must  provide  the  decision  maker  with  information 
that  informs  him  or  her  about  the  issue  at  hand.  In  most 
cases,  it  may  generate  more  work  to  examine  something 
that  arises  from  the  briefing— and  more  likely  is  due  in 
a  shorter  time  frame.  As  such,  quotations  and  saying 
about  reporting  results  were  hard  to  come  by.  We  offer 
the  following: 

•  Results: 

♦  A  mediocre  idea  well-presented  will  be  accepted 
faster  than  a  great  idea  poorly  presented. 

—  Dick  Deckro,  FS 

♦  The  very  best  analysis,  poorly  communicated,  will 
have  no  effect. 

—  Greg  Keethler 

♦  A  study  not  documented  is  a  study  not  done. 
—Anonymous 

•  Annie  Patenaude  tells  us  that  Dave  Hardison  (former 
DUSA,OR)  is  reported  to  have  said  at  the  start  of  every 
Army  Science  Board  study: 

♦  Let’s  write  up  what  we  know,  and  then  prove  it 
wrong. 

—  Hardison 
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Then  he  would  pull  out  a  draft  final  report. 

•  Complexity  of  the  report: 

♦  One  of  the  key  problems  of  contemporary  national 
security  policy  is  the  ever  widening  gap  that  has 
opened  up  between  the  sophistication  of  technical 
studies  and  the  capacity  of  an  already  overworked 
leadership  group  to  absorb  their  intricacy. 

—  Henry  Kissinger  preface  to  ES  Quade  (Rand 
R439  PR,  pg.  v) 

♦  Cy  Stanick’s  experience  with  one  project  led  him 
to  offer  that  at  the  beginning  the  team  stated:  We 
are  only  one  week  away  from  success.  A  year  later, 
the  motto  had  changed  slightly:  We  are  only  two 
weeks  away  from  success. 

•  Study  results  are  often  placed  in  the  realm  of 
doomsayers  and  there  is  a  Law  of  Doomsaying: 

♦  Predict  catastrophe  no  sooner  than  five  years 
hence  but  not  later  than  10  years  away.  Make  it 
soon  enough  to  terrify  but  distant  enough  that 
people  will  forget  if  you  are  wrong. 

—  Gregg  Easterbrook 

•  Humility  in  reporting 

♦  Be  kind,  for  everyone  you  meet  is  fighting  a 
harder  battle. 

—  Plato 

•  Phraseology: 

♦  The  difference  between  the  all-most  right  word 
and  the  right  word  is  . . .  the  difference  between 
the  lightning  bug  and  the  lightning. 

—  Mark  Twain 

Miscellaneous  Quotes 

These  are  to  be  applied  as  the  reader  sees  fit. 

•  Let  us  dare  to  read,  think,  speak,  and  write. 

—John  Adams,  1765 

•  Reading  maketh  a  full  man;  conference  a  ready  man; 
and  writing  an  exact  man. 

—Sir  Francis  Bacon 

•  Artificial  intelligence  holds  great  promise— and  always  will. 
—Clayton  Thomas 

•  The  beginning  of  wisdom  in  operations  analysis  is  that 
adequate  data  drives  useful  results  of  analysis. 
—Wayne  Hughes,  Senior  Fellow 

•  Only  the  very  strong,  or  those  so  weak  that  they  do 
not  chose  to  compete  in  terms  of  power,  can  enjoy  the 
luxury  of  acting  purely  in  the  name  of  ideals;  the  others 
have  to  make  their  compromises.  —George  Kennan, 
probably  in  the  “Long  Telegram” 


•  The  nation  that  will  insist  on  drawing  a  broad 
distinction  between  the  fighting  man  and  the  thinking 
man  is  liable  to  find  its  fighting  done  by  fools  and 
thinking  done  by  cowards. 

—Sir  William  Francis  Butler,  19th  Century  British 
soldier  and  author. 

•  There  were  gentlemen  and  there  were  seamen  in 
the  Navy  of  Charles  II.  But  the  seamen  were  not 
gentlemen,  and  the  gentlemen  were  not  seamen. 

—  Lord  Thomas  Babington  Macauley 

•  Don’t  go  into  the  clutter  alone.  You  won’t  come  out. 

—  Captain  Neil  Byrne,  Director  Tactical  Training  Group 
Pacific,  circa  1985 

•  The  Wayne  Hughes  corollary:  The  smallest  tactical 
unit  (at  sea)  is  a  pair. 

Final  Thoughts 

Roy  and  I  wrote  this  article  in  hopes  that  some  of 
its  material  can  be  used  to  improve  the  inevitable 
presentations  of  analytic  findings  and  studies.  As  Roy 
stated  in  one  of  his  many  articles,  old  analysts  sometimes 
offer  advice  to  younger  analysts  in  hopes  they  learn  and 
do  not  make  similar  mistakes  in  their  careers.  These 
quotations,  as  stated  earlier,  should  be  used  sparingly  and 
only  after  understanding  their  underlying  principles. 
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The  Iraq  and  Afghanistan  wars 
of  the  past  decade  presented 
a  number  of  highly  complex 
topics  for  the  Department 
of  Defense  (DoD)  analysis 
community.  Islamic  terrorism, 
counterinsurgency,  international 
development,  and  other  nontraditional 
challenges  left  the  community 
scrambling  at  times  for  methods 
and  approaches  to  provide  decision 
makers  with  insights  on  constantly 
evolving  problems.  As  Operation  Iraqi 
Freedom  (OIF)  came  to  a  close  and 
Operation  Enduring  Freedom  (OEF) 
began  to  wind  down,  there  was  a  brief 
moment  when  some  believed  that 
DoD  would  return  to  near-peer  and 
conventionally  oriented  challenges. 
The  resurgence  of  conflict  in  the 
Middle  East  has  made  it  clear  that  this 
will  not  be  the  case.  As  if  this  were  not 
enough,  decision  makers  also  face 
questions  about  unmanned  systems, 
cyberwarfare,  women  in  combat, 
shrinking  budgets  and  force  structure, 
military  suicides,  weapons  of  mass 
destruction,  mass  atrocity  prevention, 
cultural  awareness,  sexual  assault, 
kidnapped  Nigerian  schoolgirls,  Ebola, 
and  pivots  to  the  Pacific— to  name  but 
a  few  of  the  current  issues  of  interest. 

A  number  of  phrases  are  used  to 
describe  the  nature  of  contemporary 
challenges:  black  swans,  wicked 
problems,  social  messes,  problems 
of  the  swamp,  and  complex  systems 
(Taleb,  2010;  Rosenhead,  1992; 


Ritchey,  2013;  Snowden  and  Boone, 
2007;  Churchman,  1967).  A  complex 
system,  for  example,  is  dynamic 
with  a  whole  that  is  greater  than 
the  sum  of  its  parts.  It  has  a  large 
number  of  interacting  elements, 
nonlinear  interactions  where  minor 
changes  produce  disproportionate 
consequences,  a  history  where 
elements  evolve  together,  and 
external  conditions  and  systems  that 
constantly  change.  These  are  the 
types  of  challenges  that  DoD  decision 
makers  face  today,  and  the  analysis 
community  must  become  skilled  at 
offering  insight  and  analysis  on  these 
types  of  problems. 

The  Hard  Limits 
of  "Hard"  OR 

Within  the  field  of  operations  research 
(OR),  the  hard  limitations  of  “hard 
OR”  have  been  discussed  now  for 
several  decades.  There  was  concern 
from  within  the  field,  as  early  as  the 
1 960s,  that  OR  models  were  too 
narrow  and  failed  to  incorporate 
psychological  and  social  factors  (Kirby 
and  Rosenhead,  2005).  OR  in  the  US 
and  operational  research  in  the  UK 
during  the  1950s  and  1960s  focused 
on  “high  ground”  problems:  finding 
technical  solutions  to  “complex  but 
well-defined”  problems  that  were 
largely  about  efficiency  (Rosenhead, 
1992).  Traditional  OR  was  argued 
to  be  bested  suited  when:  1)  an 
organization  was  very  hierarchical; 

2)  few  in  the  organization  were 


analytically  sophisticated;  3)  there  was 
a  general  consensus  on  priorities;  and 
4)  the  organization’s  activities  were 
well  defined,  repetitive,  and  generated 
reliable  data.  Uncertainties  were 
dealt  with  primarily  as  mathematical 
probabilities  (Rosenhead,  1992). 
Traditional  OR  dealt  well  with  routine 
functions  such  as  logistics,  forecasting, 
manpower  planning,  and  project 
planning  (Rosenhead,  2006).  Even 
some  of  the  best  examples  of  OR 
success  in  World  War  II,  such  as  U-boat 
searches,  were  relatively  well-defined 
problems  with  agreed-upon  priorities 
and  available  quantitative  data. 

Russell  Ackoff,  a  professor  of  OR 
at  Case  Institute  of  Technology  and 
then  of  management  science  at 
Wharton,  and  a  former  president  of 
the  Operations  Research  Society  of 
America,  was  a  prominent  internal  critic 
of  OR  by  the  late  1970s  (Kirby  and 
Rosenhead,  2005).  Ackoff  argued  that 
whereas  the  nature  of  OR  was  driven 
by  the  nature  of  the  problems  it  faced 
in  the  first  two  decades  of  its  existence, 
it  became  technique  driven  rather  than 
problem  driven  (Ackoff,  1979).  As  a 
consequence,  the  “problems  that  OR 
analysts  dealt  with  were  relegated  to 
lower  levels  of  corporate  management” 
during  the  1970s  and  1980s,  since  OR 
“had  little  to  contribute  to  strategic 
planning”  (Ackoff,  1987).  In  fact,  there 
was  a  “glass  ceiling”  for  traditional  OR 
that  relegated  it  to  “tame”  problems 
(Rosenhead,  2006). 
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Jonathan  Rosenhead,  the  chair  of  OR 
at  the  London  School  of  Economics, 
wrote  of  the  “internal  crisis”  in  OR 
during  the  1970s  and  1980s  over  the 
limits  of  traditional  OR.  The  difficult 
problems,  those  most  vexing  to  an 
organization’s  top  leadership,  were 
messy  social  problems  that  defied 
technical  solutions.  They  resisted 
easy  characterization  and  there  was 
uncertainty  over  what  set  of  factors,  at 
least  for  the  moment,  were  at  the  core 
of  the  problem.  Potential  solutions 
had  to  take  diverse  interest  groups, 
as  well  as  technical  feasibility,  into 
consideration.  For  these  “problems 
of  the  swamp,”  optimization-driven 
OR  offered  “no  appropriate  handle” 
(Rosenhead,  1992). 

Alfred  Brandstein,  previously  the 
Senior  Executive  Service  (SES)  senior 
analyst  for  the  Marine  Corps,  appears 
to  have  seen  this  same  relegation  of 
OR  to  minor  problems  within  DoD  as 
well.  In  his  2004  MORS  oral  history, 
he  argued  that  OR  was  used  to  justify 
decisions  already  made  in  DoD  and 
that  “Kabuki  dance”  came  to  mind: 

“For  minor  decisions  OR  works 
great.  The  more  important  the 
decision  is,  the  less  a  role  that 
OR’s  going  to  play  in  that  decision. 
And  just  think  of  any  of  the  major 
decisions  that  have  been  made 
within  [DoD]. ..in  the  last  10,  20 
years... the  higher  the  level  military 
decision  was,  the  less  likely  that 
OR  plays  a  role.  You  want  to  know 
something  about  the  logistics  trade 
and  how  they  get  things  quicker  or 
faster  from  one  place  to  another, 
then  you  use  OR”  (MORS,  2004). 

These  remarks  were  made  a  decade 
ago,  and  it  is  even  less  clear  today 
what  contributions  OR  has  made  to 
the  most  difficult  problems  during  the 
past  decade  of  war. 


The  Rise  of  "Soft"  OR 

Although  US  OR  stayed  the  course  on 
“high  ground”  problems,  OR  outside 
the  US  took  a  different  track.  Notably, 
Ackoff  published  his  dissent  with  the 
field  of  OR  in  the  British  Journal  of 
the  Operational  Research  Society, 
and  it  was  British  OR  that  took  up 
his  challenge  of  dealing  with  wicked 
problems.  Rosenhead  remarked: 

“The  practitioner  must  choose.  Shall 
he  remain  on  the  high  ground  where 
he  can  solve  relatively  unimportant 
problems  according  to  prevailing 
standards  of  rigor,  or  shall  he 
descend  to  the  swamp  of  important 
problems  and  non-rigorous  inquiry?” 
(Rosenhead,  1992). 

In  British  OR,  the  solution  came  in 
the  form  of  “soft  OR,”  or  problem 
structuring  methods  (PSMs)  (Mingers, 
2009).  PSMs  attempt  to  assist  in 
analytic  problems  where  there  are 
multiple  actors,  differing  perspectives, 
partially  conflicting  interests, 
significant  intangibles,  and  high 
degrees  of  uncertainties  (Churchman, 
1967;  Rosenhead,  2006).  PSMs  are 
structured  but  nonmathematical,  and 
ultimately  rely  on  the  analyst  helping  to 
structure  stakeholder  and  expert  input. 
Examples  include  Checkland’s  Soft 
Systems  Methodology  (SSM),  cognitive 
mapping,  and  the  strategic  choice 
approach  (SCA)  (Mingers,  2009). 

NATO  operational  analysis  (OA) 
adopted  a  similar  approach,  with  its 
2012  release  of  the  NATO  Code  of 
Best  Practices  for  Judgement-Based 
Operational  Analysis  in  Defence 
Decision  Making  (NATO,  2012a).  The 
code  was  awarded  the  2013  NATO 
Science  and  Technology  Organization 
(STO)  Scientific  Achievement  Award. 
Judgment-based  OA  is  sometimes 
referred  to  as  “soft  OA,”  and  appears 
to  be  somewhat  broader  than  the 
PSMs  that  were  developed  by 


British  OR.  The  analyst  document 
discusses  the  “soft  OA”  method 
and  the  analyst  role  in  dealing  with 
facilitators  and  experts.  The  selected 
list  of  judgment-based  methods 
include  SSM,  SCA,  and  cognitive 
mapping;  as  well  as  causal  mapping, 
influence  diagrams,  Delphi,  analytic 
hierarchical  process  (AHP),  scenario 
building,  and  hypergames,  among 
others  (NATO,  2012b).  Although 
some  of  the  techniques  involve 
quantification,  the  commonalities  of 
this  large  class  of  methods  lie  in  the 
use  of  structured  expert  facilitation 
in  order  to  get  at  the  contours  of 
extremely  difficult  problems. 

This  raises  the  issue  of  a  new  skill  set 
that  modern  OR  analysts  require  if 
they  are  to  be  successful  at  employing 
such  methods:  understanding  and 
using  expertise.  There  is  literature  on 
experts  and  expertise  that  discusses 
the  types,  limitations,  and  appropriate 
uses  of  expertise  (Collins  and  Evans, 
2007;  Tetlock,  2006).  There  are  also 
a  number  of  practical  considerations, 
such  as  the  identification,  recruitment, 
facilitation,  and  logistics  of  employing 
experts.  It  is  beyond  the  scope  of  this 
short  article  to  discuss  these  matters, 
but  they  are  significant  given  the 
prominence  that  PSMs  and  judgment- 
based  methods  give  to  expert  and 
stakeholder  input. 

Multidisciplinary:  The 
Missing  Piece 

To  address  the  complex  contemporary 
challenges,  the  analysis  community 
should  once  again  become  a  more 
multidisciplinary  community.  The 
multidisciplinary  roots  of  OR  are  often 
invoked  but  rarely  replicated:  two 
of  the  forefathers  of  American  OR, 
Philip  Morse  and  George  Kimball, 
recruited  operations  researchers 
during  World  War  II  from  physicists, 
chemists,  mathematicians,  biologists, 
psychologists,  and  economists 
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Table  1.  National  Science  Foundation  (NSF)  Survey  of 
Earned  Doctorates,  201 2. 


Field 

Number  of  Subfields 

Doctorates  in  2012 

Life  sciences 

81 

12,045 

Physical  sciences 

61 

8,952 

Psychology  and  social  sciences  40 

8,353 

Engineering 

30 

8,427 

Education 

42 

4,802 

Humanities 

49 

5,503 

Other 

30 

2,926 

(Morse  and  Kimball,  1951).  Returning 
to  multidisciplinary  OR,  your 
grandfather’s  OR,  is  vital  now  for  the 
community.  As  analytic  challenges 
become  more  and  more  complex  and 
begin  to  span  multiple  disciplines, 
there  is  an  urgent  need  for  the 
analysis  community  to  be  able  to 
meaningfully  access  knowledge  and 
methods  from  across  disciplines  in 
order  to  be  relevant.  Although  this 
will  not  prevent  the  community  from 
being  surprised  by  new  events  and 
challenges,  having  diverse  intellectual 
backgrounds  among  its  members 
would  allow  it  to  start  examining  new 
challenges  with  a  greater  sum  of 
collective  knowledge. 

Ackoff  argued  in  1 979  that  the  rise 
of  OR  professional  societies  and 
OR  academic  programs  effectively 
stripped  OR  of  any  vestiges  of  its 
multidisciplinary  past,  and  that  he 
shared  in  the  blame  as  a  previous 
head  of  an  OR  professional  society 
and  OR  faculty  member  (Ackoff, 

1979).  Yet  we  have  recent  examples  of 
what  multidisciplinary  reach  can  bring 
to  the  table.  One  2009  RAND  study, 
sponsored  by  Mr.  Jim  Bexfield  at  what 
was  then  the  Office  of  the  Secretary 
of  Defense  (OSD)  Program  Analysis 
&  Evaluation  (PA&E),  evaluated  the 
literature  on  terrorism  across  multiple 
fields  of  study  (Davis  and  Cragin, 
2009).  The  resulting  report  illustrates 
the  multidisciplinary  capability 
required  even  to  review  the  existing 
research  on  a  complex  problem  such 
as  terrorism. 

When  asked  during  his  MORS 
oral  history  whether  the  analysis 
community  should  be  taking  the 
multidisciplinary  approach  on  one 
hand,  or  the  OR  curriculum  approach 
on  the  other,  Brandstein  responded: 

“Definitely  multidisciplinary ....  You 
need  math.  That’s  clear.  And  you 


need  to  have  a  liberal  arts  education 
perhaps  in  order  to  think.  You  may 
have  one  or  two  OR  courses,  but 
if  you  have  the  math  you  can  pick 
up  the  rest.  If  you  don’t  have  the 
sociology  background  or  human 
psychology  background  or  at  least 
know  what’s  going  on  in  those  fields, 
it’s  much  more  difficult  to  pick  up. 
That’s  the  way  our  OR  forefathers 
were”  (MORS  2004). 

The  2012  results  from  the  annual 
“Survey  of  Earned  Doctorates”  in 
Table  1  shows  the  number  of  major 
academic  fields  and  subfields  that 
exist  in  the  US  today.  There  are  333 
listed  subfields  of  study.  Subtracting 
the  three  times  that  OR  manages  to 
be  listed,  this  leaves  330  subfields 
outside  of  OR  (NSF,  2012). 

How  many  of  these  330  other 
subfields  can  the  analysis  community 
meaningfully  access?  The  current 
community  has  access  to  OR, 
statistics,  applied  math,  and  closely 
related  fields,  but  is  still  extremely 
narrow  in  its  overall  range.  Neither 
should  the  community  dismiss 
humanities:  they  include  Middle 
Eastern  languages,  Middle  Eastern 
history,  and  religion.  The  last  decade 
underscores  their  significance. 

The  “other”  field  listed  in  the  NSF 
table  includes  communications  and 


management,  subfields  that  are  also 
helpful  in  the  understanding  of  modern 
military  operations.  The  spread  of 
quantitative  methods  into  a  whole 
range  of  fields  also  makes  it  perfectly 
possible  to  hire  analysts  for  the  1515 
series  from  a  vast  number  of  subfields, 
while  going  well  beyond  the  modest 
math  and  science  requirements  set  for 
that  occupational  field. 

At  the  same  time,  because  of  changes 
in  higher  education,  there  has  never 
been  a  better  time  for  the  defense 
community  to  hire  from  a  broad  range 
of  disciplines.  The  increasing  number 
of  PhDs,  declining  number  of  tenure- 
track  faculty  positions,  and  increasing 
numbers  of  adjuncts  (estimated  to 
be  70  percent  of  college  instructors) 
has  created  a  large  number  of  highly 
educated  individuals  that  could  be 
recruited  into  the  analysis  community 
(June,  2014). 

A  Practical  Example 

Combining  the  multidisciplinary 
approach,  methods  developed  for 
wicked  problems,  and  a  practical 
understanding  of  experts  and 
expertise  is  not  a  hypothetical 
proposition.  This  has  existed  in  areas 
such  as  management  consulting 
and  European  defense  analysis— in 
some  cases  for  decades.  Closer  to 
home,  the  Marine  Corps  Operations 
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Analysis  Division  (OAD)  used  this 
exact  recipe  for  the  Joint  Irregular 
Warfare  Analytic  Baseline  (JIWAB) 
study.  JIWAB  was  a  three-year  effort 
that  departed  from  the  modeling  and 
simulation  (M&S)-based  approach  that 
dominated  irregular  warfare  (IW)  in  the 
analysis  community  during  the  height 
of  the  Iraq  and  Afghanistan  wars.  It 
was  a  response  to  Dr.  George  Akst’s 
direction  to  OAD  to  find  an  alternative 
to  the  M&S-dominated  methods  used 
for  conventional  campaign  modeling. 
Although  developed  outside  the 
regular  Support  to  Strategic  Analyses 
(SSA)  process,  it  produced  an  SSA- 
affiliated  scenario,  interagency 
CONOPS,  wargames,  and  a  set  of 
capability  analyses. 

One  JIWAB  hallmark  was  a  high 
level  of  collaboration  from  the  very 
beginning  with  knowledgeable 
interagency  participants,  recognized 
regional  experts,  conflict  experts, 
and  skilled  facilitators  in  the  PSMs 
and  other  methods  used  in  the  study. 
Participants  in  the  study  came  from 
five  different  Department  of  State 
(DoS)  offices,  five  different  US  Agency 
for  International  Development  (USAID) 
offices,  US  Department  of  Agriculture 
(USDA),  US  Institute  of  Peace  (USIP), 
four  separate  nonuniversity  research 
centers,  three  nongovernmental 
organizations  (NGOs),  faculty  from  10 
different  universities,  and  a  similarly 
lengthy  list  of  DoD  organizations. 

However,  simply  being  able  to 
list  a  large  number  of  outside 
participants  is  not  the  correct 
metric  of  success.  The  real  test  of 
meaningful  collaboration  is  the  extent 
to  which  outside  participants  also 
feel  they  gain  from  the  process  and 
the  extent  to  which  their  input  shape 
the  final  products.  This  often  takes 
the  form  of  requests  for  the  study 
products,  positive  referrals  from 
others  in  their  organization  who  had 


previously  participated,  requests  to 
disseminate  study  results  to  a  broader 
audience,  and  feedback  on  their  own 
new  insights  gained  as  a  result  of 
participating  in  the  study.  In  JIWAB’s 
case,  this  included  requests  from  DoS 
and  USAID  for  study  products  ahead 
of  strategy  and  policy  decisions  for 
the  geography  in  question. 

Similarly,  a  study’s  analytic  rigor  or 
purported  innovations  are  not  self- 
assessed,  but  validated  through  the 
feedback  and  responses  from  those 
already  recognized  as  experts  in  the 
methods  used.  The  highest  praise 
for  JIWAB’s  analysis  came  from  one 
methodology  expert  who  said  that 
the  team  had  taken  his  method  and 
“ratcheted  up  the  rigor.”  Another, 
a  wargaming  expert,  called  a  new 
proposed  method  of  wargame 
analysis  “a  breakthrough”  and  JIWAB 
team  members  “pioneering.”  The 
professional  PSM  facilitators  asked 
team  members  to  coauthor  papers, 
resulting  in  publications  in  peer- 
reviewed  journals.  The  developer  of 
another  method  who  quietly  adopted 
the  adaptations  that  the  team  had 
made  for  JIWAB.  In  short,  the  mere 
adoption  of  new  methods  is  not 
enough:  there  should  be  testimony  of 
their  skillful  application  by  others  who 
actively  practice  the  methods. 

Conclusion 

The  analysis  community  is  facing 
an  era  of  more  and  more  complex 
analytic  challenges.  To  stay 
relevant  at  a  minimum  and  to  thrive 
if  conditions  are  right,  there  is  a 
pressing  need  for  the  community 
to  become  more  multidisciplinary, 
to  adopt  established  methods  from 
OR  and  other  disciplines  designed 
specifically  for  wicked  problems,  and 
to  become  deft  at  collaborating  with 
a  very  wide  range  of  experts.  This 
would  require  an  active  strategy  to 
acquire  multidisciplinary  capability, 


a  deliberate  effort  to  develop  analyst 
skills  in  expert-driven  methods, 
and  conscious  management  of 
organizational  cultures  that  gives 
subconscious,  automatic  primacy  to 
traditional  OR  methods.  To  illustrate 
the  power  of  organizational  culture 
over  professed  mission  statements: 
MORS  in  2009  released  a  five-year 
plan  stating  that  the  society  would 
no  longer  “limit  topics  to  traditional 
operations  research”  but  reach  out 
to  other  disciplines  (MORS,  2009). 

Yet  it  unfailingly  awarded  nine  out  of 
1 0  Barchi  and  Rist  prizes  from  2009 
to  2013  to  analysts  using  traditional 
OR  methods  such  as  optimization, 
forecasting,  and  M&S. 

Many  challenges  currently  stand 
in  the  way  of  this  proposed  path: 
a  community  culture  that  defines 
analysis  and  analysts  extremely 
narrowly,  education  pipelines  and 
hiring  practices  that  institutionalize  a 
relatively  few,  dominant  sources  for 
new  analysts  rather  than  encouraging 
diversification  of  academic 
backgrounds,  and  the  general 
risk-aversion  that  characterizes 
bureaucracies  (Wilson,  1989). 
Bureaucracies  are  designed  to  resist 
innovation.  When  they  do  take  on  new 
tasks,  such  as  the  analysis  of  wicked 
problems  using  methods  outside 
traditional  OR,  these  are  often  seen  as 
peripheral  tasks.  Yet  real  innovation 
happens  when  core  tasks,  rather  than 
peripheral  ones,  change.  The  longer 
agencies  exist,  the  harder  and  more 
costly  it  is  to  change  their  core  tasks. 
This,  noted  the  late  political  scientist 
James  Q.  Wilson,  is  why  bureaucratic 
innovation  is  rare  (Wilson,  1989). 

The  analysis  community  at  this 
moment  has  an  opportunity  to  take 
stock  of  its  future  and  its  past.  It 
faces  both  a  generational  turnover 
in  its  ranks,  and  new  pressures  to 
show  relevance  in  bringing  analytic 
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insight  to  difficult  problems.  Can  the 
pressures  to  adapt  in  meaningful  ways 
overcome  institutional  inertia  and  the 
comfort  of  the  familiar?  Its  choices 
now  will  set  the  tone  for  another 
generation  of  analysts. 
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MORS  Heritage  Pa9^002_2005 

As  we  continue  the  countdown  to  the  50th  Anniversary  of  MORS,  we  would  like  to  revisit  our  proud  history  and  highlight 
the  past  leaders  of  the  Society  and  key  accomplishments  over  those  years.  Each  edition  of  Phalanx  will  provide  insight 
into  several  years  of  history.  Enjoy  reading  about  these  individuals  and  what  they  have  accomplished.  More  information 
on  the  Past  Presidents  (PP)  can  be  found  on  the  MORS  website,  including  their  oral  histories. 


Elected  Directors  (consolidated  list  2002-2005) 


Dr.  Thomas  Allen,  FS 

Priscilla  Glasow,  FS 

Terry  McKearney 

COL  Jeffrey  Appleget 

Dr.  Niki  Goerger,  FS 

Patrick  McKenna,  FS 

Donald  Bates 

Dr.  Richard  Hayes 

Kirk  Michealson,  FS 

Lt  Col  Suzanne  Beers,  FS 

Robert  Holcomb 

Anne  Patenaude,  FS 

Dr.  Ted  Bennett 

Susan  Iwanski,  FS 

Dr.  Steven  Pilnick 

Joseph  Bonnet 

Jack  Keane,  FS 

Mark  Reid 

Denis  Clements 

Gregory  Keethler,  FS 

Dr.  Roy  Rice,  FS 

Dr.  W.  Forrest  Crain 

William  Kroshl 

Richard  Rosenthal 

Col  Gerald  Diaz 

LTC  Michael  Kwinn 

Dr.  Robert  S.  Sheldon,  FS 

Lawrence  Dick,  FS 

Michael  Leonard,  FS 

Cortez  Stephens 

William  Dunn,  FS 

Dr.  Andrew  Loerch,  FS 

COL  George  Stone  III 

Robert  Eberth 

John  Marriott 

Edward  “Ted”  Smyth,  FS 

Helaine  Elderkin,  FS 

Dr.  Willie  McFadden,  III,  FS 

Charles  Taylor 

John  Ferguson 

COL  Michael  McGinnis,  FS 

Corinne  Wallshein 

W.  Dean  Free 

Lana  McGlynn 

Lt  Col  Kirk  Yost 

Michael  Garrambone 

LtCol  Gregory  McIntyre 

Sponsors 

US  Army 

Deputy  Under  Secretary  of  the  Army 
(Operations  Research) 

Mr.  Walter  W.  Hollis,  FS 
Sponsor’s  Rep:  COL  Hoa  Generazio 

US  Navy 

Director  Assessment  Division  (OCNO) 

Mr.  Gregory  Melcher 
Sponsor’s  Rep:  Mr.  Herbert  Cupo 

US  Marine  Corps 

Commanding  General 

Marine  Corps  Combat  Development  Center 

LtGen  Edward  Hanlon,  Jr. 

Sponsor’s  Rep:  Col  Philip  J.  Exner/Dr.  George  Akst 


Joint  Staff 

Director  for  Force  Structure,  Resources  and  Assessment  (J8) 
Lt  Gen  James  Cartwright 
Sponsor’s  Rep:  Mr.  Peter  Byrne 

US  Air  Force 

Director  Air  Force  Studies  and  Analyses  Agency 
Dr.  Jacqueline  Henningsen 
Sponsor’s  Rep:  Mr.  Royce  Reiss 

Office,  Secretary  of  Defense 

Office  of  the  Secretary  of  Defense,  Program  Analysis 
and  Evaluation 
Mr.  Eric  Coulter 

Sponsor’s  Rep:  Mr.  James  Bexfield,  FS 
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MORS  Staff 

Brian  Engler,  FS, 

Executive  Vice  President 
Natalie  S.  Kelly,  Vice  President, 
Administration 
Cynthia  Kee, 

Administrator 
Corrina  Ross-Witkowski, 
Communications  Manager 

Significant  Events 

August  2002:  MORS  President, 
Edward  Smyth,  represented  MORS 
at  the  19th  International  Symposium 
of  Military  Operational  Research 
(ISMOR)  conducted  at  Eynsham 
Hall  near  Oxford,  UK.  In  addition  to 
briefing  ISMOR  on  MORS  activities 
an  invitation  was  extended  for 
ISMOR  members  to  publish  in 
Phalanx  and  to  attend  unclassified 
MORS  special  meetings. 

September  2002:  The  MORS 
office  moved  from  the  Duke  Street 
location  to  the  office  suite  on  N. 
Beauregard  Street. 

2002:  MORS  also  gained  permission 
from  UK  officials  to  republish  a  book 
entitled  Operational  Research  in  the 
RAF  as  another  volume  in  the  MORS 
Heritage  Series  of  publications. 

2002:  The  educational  forum  went 
“on  the  road”  for  the  first  time  and 
was  held  at  NPS  with  a  number 
of  West  Coast  civilian  schools  in 
attendance.  Mike  Garrambone  served 
as  the  program  chair  for  the  event 
that  featured  a  team  competition  for 
students  in  attendance. 

2002-2003:  Formed  an  ad  hoc 
committee  of  the  BoD  under  the 
leadership  of  Lana  McGlynn  to 
review  the  existing  composite  group/ 
working  group  structure.  Changes  to 
the  composite  group/working  group 
structure  were  implemented  during 
the  72nd  MORSS  in  2004. 


Javarline  Nelson, 

Administrative  Assistant 
Helaine  Elderkin,  FS, 
Counsel 
Paul  West, 

Editor,  Phalanx 


2002-2003:  Implemented  the  sponsor 
hot  topics  segment  of  the  annual 
symposium  beginning  with  the  71st 
MORSS. 

2003:  Implemented  the  Sponsor’s 
Corner  in  Phalanx  in  which  the 
MORS  sponsors  wrote  articles  on 
various  topics,  starting  with  Jackie 
Henningsen  in  June  2003. 

2003:  During  the  71st  Plenary,  MORS 
President  signaled  the  need  for  MORS 
to  engage  with  the  then  newly  created 
Department  of  Homeland  Security 
and  to  pursue  that  department  as  a 
potential  MORS  sponsor. 

2005:  MORS  Executive  Vice 
President,  Brian  Engler,  represented 
MORS  at  the  22nd  International 
Symposium  of  Military  Operational 
Research  (ISMOR)  conducted  at 
New  Place,  near  Bishop  Waltham, 
Hampshire,  UK. 

71st  MORS  Symposium,  Marine 
Corps  Base,  Quantico,  Virginia,  June 
10-12,  2003.  Military  Operations 
Research  for  Tomorrow’s  Warrior. 

Dr.  Roy  E.  Rice,  FS  and  Dr.  Robert 
S.  Sheldon,  FS  were  inducted  as 
Fellows  of  the  Society.  Natalie 
Crawford  received  the  Wanner 
Award.  Joseph  Bolmarcich  received 
the  Thomas  Award.  Thomas  Lucas 
and  William  Vinyard  received  the 
Walker  Award.  Elisabeth  Pate- 
Cornell  and  Seth  Guikema  received 
the  MOR  Journal  Award  for 


Dr.  Richard  Deckro, 

Editor,  Military  Operations 
Research  (MOR) 

W.  Addison  Woods, 
Database  Consultant 


“Probabilistic  Modeling  of  Terrorist 
Threats:  A  Systems  Analysis 
Approach  to  Setting  Priorities 
among  Countermeasures,”  Military 
Operations  Research,  Volume  7, 
Number  4. 

72nd  MORS  Symposium,  US 

Naval  Postgraduate  School, 
Monterey,  California,  June  22-24, 
2004.  OR  Support  for  Tomorrow’s 
Force.  Michael  F.  Bauman,  FS  and 
Dr.  Priscilla  A.  Glasow,  FS  were 
inducted  as  Fellows  of  the  Society. 
David  M.  Maddox  received  the 
Wanner  Award.  Stuart  H.  Starr,  FS 
received  the  Thomas  Award.  MAJ 
Rob  Kewley  and  LTC  Larry  Larimer 
received  the  Walker  Award.  David 
P.  Kierstead  and  Donald  R.  Del 
Balzo  received  the  MOR  Journal 
Award  for  “A  Genetic  Algorithm 
Applied  to  Planning  Search  Paths  in 
Complicated  Environments,”  Military 
Operations  Research,  Volume  8, 
Number  2. 

73rd  MORS  Symposium,  US  Military 
Academy,  West  Point,  New  York, 

June  21-23,  2005.  Balancing  Risk  for 
an  Uncertain  Future.  The  following 
individuals  were  inducted  as  Fellows 
of  the  Society:  Dr.  Thomas  L.  Allen, 

FS;  Dennis  R.  Baer,  FS  and  Edward 
A.  Smyth,  FS.  Vernon  M.  Bettencourt, 
FS  received  the  Wanner  Award.  Alan 
R.  Washburn  received  the  Thomas 
Award.  Peter  A.  Davidson  received 
the  Walker  Award.  Seethal  Mishra, 
Rajan  Batta,  and  Robert  J.  Szczerba 
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received  the  MOR  Journal  Award  for  “A  Rule  Based  Approach  for  Aircraft  Dispatching  to  Emerging  Targets,”  Military 
Operations  Research,  Volume  9,  Number  3. 

MORS  Presidents 

37th  MORS  President:  Edward  "Ted"  Smyth,  2002-2003 

Edward  Smyth  served  as  Secretary  in  1999-2000,  Vice  President  for  Professional  Affairs  in 
2000-2001,  and  President-Elect  in  2001-2002.  He  served  as  MORS  President  in  2002-2003.  He 
was  elected  a  Fellow  of  the  Society  in  2005. 

Mr.  Smyth  is  a  1966  graduate  of  the  US  Naval  Academy  with  a  BS  in  engineering  and  a  major 
in  history.  He  subsequently  received  an  MS  in  operations  research  in  1 970  from  the  US  Naval 
Postgraduate  School  and  an  MA  in  history  from  Old  Dominion  University  in  1975. 

He  is  a  former  Marine  Corps  Colonel  with  30  years  of  active  service  during  which  he  commanded  Marine  Corps  units  at 
the  company/battery,  battalion,  and  regimental  level.  During  his  Marine  Corps  career  his  primary  occupational  specialty 
was  artillery/fire  support  with  subspecialties  as  a  military  operations  analyst  and  historian.  As  an  operations  analyst  he 
held  a  number  of  varied  assignments,  including  service  as  an  operations  analyst  at  Headquarters,  Fleet  Marine  Force 
Atlantic,  Norfolk,  Virginia;  an  assistant  professor  of  operations  research,  US  Naval  Academy,  Annapolis,  Maryland;  the 
senior  Naval  operations  analyst,  Combined  Forces  Command,  Seoul,  Republic  of  Korea;  professor  of  Naval  science  at 
Norwich  University;  and  Director,  Marine  Corps  Studies  and  Analysis  Division,  Ouantico,  Virginia. 

Mr.  Smyth  possesses  considerable  teaching  experience  in  the  use  of  quantitative  methods,  operations  research, 
and  systems  engineering  and  has  served  on  the  faculties  of  the  University  of  Virginia,  US  Naval  Academy,  George 
Washington  University,  University  of  Maryland,  Webster  University,  Norwich  University,  and  Johns  Hopkins  University.  He 
is  also  the  coauthor  of  Naval  Operations  Analysis  (US  Naval  Institute  Press,  Annapolis,  Maryland,  1 977). 

Since  joining  JHU/APL  in  1995  he  has  served  as  Director,  Campaign  Analysis  Team  of  the  Surface  Combatant  21 
Cost  and  Operational  Effectiveness  Analysis,  employing  campaign-level  simulations  to  assist  in  discriminating  among 
surface  combatant  alternatives.  He  has  coordinated  efforts  in  support  of  OPNAV  sea  strike  analyses,  served  as  the 
technical  lead  of  the  Advanced  Land  Attack  Missile  AoA,  provided  analytical  support  to  the  deputy  assistant  secretary 
of  the  Navy  for  littoral  and  mine  warfare,  developed  and  organized  a  three-day  symposium  sponsored  by  MORS  on 
the  subject  of  “Analysis  of  Urban  Warfare,”  and  has  served  as  the  senior  JHU/APL  analyst  in  support  of  the  National 
Security  Agency’s  Signals  Intelligence  Requirements  Office.  He  is  currently  the  technical  branch  supervisor  in  the 
National  Security  Analysis  Department. 

38th  MORS  President:  Dr.  Willie  McFadden  II,  2003-2004 


Dr.  Willie  McFadden  served  as  Vice  President  Professional  Affairs  in  2001-2002  and  President- 
Elect  in  2002-2003.  He  was  President  of  MORS  in  2003-2004  and  was  elected  a  Fellow  of  the 
Society  in  2008. 

Dr.  McFadden  earned  a  BS  degree  in  engineering  from  the  United  States  Military  Academy  at 
West  Point,  an  MS  degree  in  operations  research  from  the  Naval  Postgraduate  School,  and  a  PhD 
in  engineering  management  from  Old  Dominion  University. 

Dr.  McFadden  also  served  23  years  active  duty  in  the  US  Army.  He  was  commissioned  in  the  field  artillery  and  has  served 
as  Commander,  Headquarters  &  Headquarters  Battery  VII  Corps  Artillery  at  Augsburg,  Germany  and  Southwest  Asia 
and  Commander,  C  Battery  1st  Battalion  36th  Field  Artillery  Regiment,  17th  Field  Artillery  Brigade,  VII  Corps  Artillery  at 
Augsburg,  Germany.  As  an  operations  analyst,  he  has  filled  assignments  at  the  Joint  Venture  Task  Force  XXI  Operations 
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&  Organization  Division,  Office  of  the  Deputy  Chief  of  Staff  for  Combat  Developments  and  Plans,  Analysis  and 
Evaluation,  Office  of  the  Deputy  Chief  of  Staff  for  Resource  Management,  Training  and  Doctrine  Command,  Ft  Monroe, 
Virginia.  Additionally,  Willie  served  as  a  permanent  military  professor  at  the  United  States  Military  Academy,  West  Point, 
New  York.  While  there,  he  served  as  the  Engineering  Management  Program  Director  for  the  Department  of  Systems 
Engineering  and  the  director  of  the  Lifecycle  Acquisition  Management  Institute  at  the  Academy.  Willie  was  promoted  to 
the  rank  of  professor  of  engineering  management  while  serving  as  a  tenured  military  professor  at  the  Academy. 

Dr.  McFadden  is  currently  the  director  of  Integrated  Systems  Frameworks  at  Teledyne  Brown  Engineering.  Fie  leads 
and  manages  the  objective  simulation  framework  (OSF)  contract  and  program  for  TBE  and  provides  engineering 
leadership  throughout  the  company.  Before  joining  Teledyne  Brown,  he  served  as  director  of  Army  programs  at  the  ARES 
Corporation  from  2010-2012,  where  he  provided  strategic  direction  and  planning  to  support  ARES’s  growth  through 
business  opportunities.  As  a  senior  associate  at  Booz  Allen  Hamilton  from  2006-2010,  Dr.  McFadden  was  responsible  for 
developing  the  vision  and  strategy  to  capture  multiple  opportunities  in  Missile  Defense  Agency  (MDA),  Space  and  Missile 
Defense  Command  (SMDC),  Civil,  and  Aviation  and  Missile  Research,  Development,  and  Engineering  Center  (AMRDEC) 
elements  and  leading  the  Booz  Allen  Huntsville  office  modeling  simulation,  wargaming,  and  analysis  (MSW&A)  capability 
offering  and  management  and  leadership  of  the  MSW&A  team  members. 

Dr.  McFadden  is  also  Past-President  and  Fellow  of  the  Society  for  the  American  Society  for  Engineering 
Management  (ASEM). 

39th  MORS  President:  Dr.  Andrew  Loerch,  2004-2005 

Dr.  Andrew  Loerch  served  as  Vice  President  Finance  and  Management  in  2002-2003  and 
President-Elect  from  2003-2004.  He  was  President  of  MORS  from  2004-2005  and  elected  a 
Fellow  of  the  Society  in  2008. 

Dr.  Loerch  has  a  BS  in  mechanical  engineering  from  the  Polytechnic  Institute  of  Brooklyn,  an  MS 
in  operations  research  from  the  Naval  Postgraduate  School,  and  a  PhD  in  operations  research 
from  Cornell  University. 

He  is  currently  an  associate  professor  in  George  Mason  University’s  Department  of  Systems  Engineering  and  Operations 
Research.  Andy  joined  George  Mason  University  in  2000,  following  a  26-year  career  in  the  US  Army.  At  GMU,  he 
directs  the  operations  research  subprogram  and  teaches  and  performs  research  in  operations  research  methods  and 
applications.  During  his  Army  career,  Andy  held  numerous  operations  research  analyst  positions  and  represented  the 
Army  in  several  joint  studies.  His  last  position  was  Chief,  Force  Strategy  Division,  Center  for  Army  Analysis.  Andy  retired 
with  the  rank  of  colonel. 

Dr.  Loerch  was  the  recipient  of  the  Dr.  Wilbur  B.  Payne  Memorial  Award  for  Excellence  in  Analysis  (three  awards)  and 
was  named  LMI’s  inaugural  Distinguished  Visiting  Scholar. 
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Review  Are  We  Winning? 


A  Brief  History  of  Military 
Operations  Assessment 


Dr.  Adam  Shilling 

Operations  Analysis  Division,  Center  for  Army  Analysis 
adam.p.shilling.civ@mail.mil 


Mr.  Cortez  "Steve"  Stephens 

USMC  Operations  Analysis  Division,  Combat  Development  &  Integration 
cortez.stephens@usmc.mil 


ne  of  the  intriguing  aspects 
of  Ben  Connable’s  book 
Embracing  the  Fog  of  War: 
Assessment  and  Metrics  in 
Counterinsurgency  (Rand 
Corp.,  2012;  http://www. 
rand.org/pubs/monographs/MG1086. 
html),  especially  to  those  of  us 
grappling  with  how  to  do  operations 
assessments,  was  the  chapter  on 
assessments  during  the  Vietnam  War. 

It  put  what  we  were  doing  under  a 
different  light.  We  were  not  doing,  as 
many  of  us  felt,  something  that  had 
never  been  done  before.  There  was 
historical  precedent. 

Are  We  Winning?  A  Brief  History 
of  Military  Operations  Assessment, 
by  Emily  Mushen  and  Jonathan 
Schroden  (http://www.cna.org/ 
research/201 4/are-we-winning), 
published  in  August  2014,  is  just  as 
intriguing.  Are  We  Winning  sets  forth 
a  historical  continuum  and  takes  the 
reader  from  Vietnam  through  the  Cold 
War  and  the  following  Information 
Age  to  the  current  post-9/1 1  period. 
Mushen  and  Schroden  state  that  one 
of  the  purposes  of  Are  We  Winning  is 
to  serve  as  a  foundation  and  source  of 


encouragement  for  the  research  and 
production  of  a  more  complete  history 
of  operations  assessment. 

The  authors  are  highly  qualified  to 
report  on  these  issues.  Mushen  has 
deployed  to  Al  Anbar  multiple  times 
and  has  worked  as  an  analyst  in  both 
Iraq  and  Afghanistan.  Schroden  is  also 
a  veteran  of  multiple  deployments 
to  Iraq  and  Afghanistan,  and  an 
author  of  several  journal  articles 
on  assessment  issues.  Mushen 
and  Schroden  trace  operations 
assessment  back  to  the  roots  of 
operations  analysis  in  World  War  II. 
They  consider  two  historical  periods, 
the  Cold  War  and  the  post-Cold  War 
Information  Age,  as  significant  to  the 
evolution  of  operations  assessment. 
Additionally,  Are  We  Winning  explains 
the  status  of  operations  assessment 
at  the  beginning  and  end  of  several 
conflict  periods  (starting  from  the 
outset  of  the  Vietnam  War)  and 
points  out  factors  that  have  affected 
the  development  of  operations 
assessment  during  each  period.  Their 
perspectives  are  thought-provoking 
and  definitely  set  the  case  for  further 
historical  research. 


The  authors  term  the  history  of 
assessment  as  an  “oscillation” 
between  the  primacy  of  quantitative 
measures  and  primacy  of  qualitative 
judgments.  The  rise  of  effects-based 
operations  in  the  late  1980s  and  the 
advent  of  information  technologies 
led  to  expectations  that  assessors 
could  gather,  organize,  and  analyze 
the  quantitative  data  necessary  to 
fuel  a  comprehensive  understanding 
of  even  complex  environments. 

This  swung  the  pendulum  toward 
quantitative  primacy,  but  experience 
in  Afghanistan  has  proven  that  a  too- 
quantitative  schema  has  inevitably 
missed  something  important  or 
aggregated  away  the  understanding 
that  assessors  and  commanders 
were  seeking.  The  numbers,  without 
informed  interpretation,  have  lost 
the  context  and  nuance  critical  to 
understanding  what  the  authors 
term  “highly  relevant  complexities” 
of  the  operational  environment, 
and  “obscured  underlying  causes 
and  effects.”  The  good  news 
is  that  emerging  doctrine  has 
explicitly  recognized  the  inherently 
qualitative  nature  of  assessment  by 
acknowledging  that  human  judgment 
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is  required  to  make  sense  of  the 
hard  data. 

The  idea  that  assessment’s  purpose 
is  to  “measure  progress”  appears 
throughout  the  history.  Although  it 
is  probably  not  accurate  to  say  the 
authors  advocate  this  point  of  view, 
it  may  be  accurate  to  say  that  this 
point  of  view  is  an  error  prevalent 
in  the  assessment  community,  but 
one  that  we  have  recognized  and  are 
correcting.  Assessment  does  provide 
a  measure  of  progress  in  operations 
and  campaigns,  but  that  is  not  its 
purpose.  The  knowledge  of  “where 
we  are”  in  an  operation  or  campaign 
is  of  little  value  unless  commanders 
are  using  that  information  to  sustain, 
modify  or  terminate  activities  to 
achieve  objectives.  Therefore, 
assessment’s  purpose,  stated 
explicitly  in  emerging  assessment 
publications,  is  to  “make  operations 
more  effective.” 

The  authors  also  posit  that 
assessment  must  serve  two  “often 
competing”  audiences.  The  “external” 
audience  is  policymakers  and  senior 
leadership,  and  assessment  serves 
a  secondary  purpose  of  providing 
accountability  to  these  groups  so 
they  can  determine  whether  to 
support  a  theater  commander’s 


acknowledge  that  a  senior  commander 
may  use  assessment  products  as 
part  of  “messaging  efforts”  to  “bolster 
the  position  he  intends  to  defend” 
with  policymakers.  The  authors 
acknowledge  that  this  might  have 
implications  for  the  integrity  of  the 
assessment.  The  “internal”  audience 
is  the  unit  and  its  subordinates.  These 
folks  are  interested  ultimately  in 
achieving  their  objectives  effectively, 
and  assessment  serves  its  primary 
purpose  of  making  operations 
more  effective.  The  integrity  of  the 
assessment  is  critical  to  this  goal. 

The  art  of  operations  assessment 
for  21st  Century  warfare  is  still 
developing  and,  as  Mushen  and 
Schroden  point  out,  warrants 
further  study.  By  surveying  trends 
and  developments  in  operations 
assessments  over  the  years,  Are  We 
Winning  serves  as  an  introduction  to 
operations  assessment  for  readers 
from  both  inside  and  outside  of  the 
military  community  who  are  unfamiliar 
with  operations  assessment.  At  the 
same  time,  it  provides  valuable  insight 
to  those  who  are  generally  familiar 
with  operations  assessment  as  well  as 
those  who  are  grappling  with  how  to 
do  operations  assessments. 

Those  of  us  still  grappling  with  how  to 


focusing  on  the  next  fight.  Our  task 
is  to  consider  where  we  are  today 
with  regard  to  operations  assessment 
in  order  to  determine  where  we 
should  be  going  tomorrow.  Many 
analysts  have  worked  on  operations 
assessments  during  the  post-9/1 1 
period,  and  a  clearer  picture  of 
where  we  are  today  on  operations 
assessment  seems  to  be  taking  shape 
out  of  our  collective  experiences. 
Taking  it  one  step  further,  realizing 
where  we  were,  yesterday  and  in  the 
years  previous,  is  equally  essential 
to  determining  where  we  should  be 
going  with  operations  assessment. 
This,  unfortunately,  is  out  of  the 
realm  of  our  collective  experiences. 
This  is  why  Are  We  Winning  is  so 
intriguing.  Operations  assessment  has 
a  history,  and  Are  We  Winning  causes 
us  to  understand  that  it  is  a  largely 
unwritten  history. 

Dr.  Adam  Shilling  is  the  Center  for 
Army  Analysis  representative  to  efforts 
to  revise  joint  assessment  doctrine. 

He  is  a  veteran  of  deployments  to 
Iraq,  Afghanistan,  and  the  Horn  of 
Africa  as  both  a  serviceman  and 
civilian  assessor. 

Mr.  Steve  Stephens  has  worked 
in  assessment  cells  in  Iraq  and 
Afghanistan. 
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&  Thinking  Analytically 
BYJOHNTOCZEK 


Four  different  fighters  are  having  an  all-out  battle  to 
determine  who  among  them  is  the  strongest.  The  image 
above  shows  those  four  fighters:  Allan,  Barry,  Charles, 
and  Dan. 

Each  fighter  has  varying  attack  and  health  abilities.  At 
the  start  of  the  battle,  they  have  differing  health  points: 
Allan  has  10,  Barry  has  12,  Charles  has  16,  and  Dan 
has  1 8.  Also,  each  fighter  has  differing  attack  points: 
Allan  has  4,  Barry  has  3,  Charles  has  2,  and  Dan  has  1 . 

The  battle  takes  place  over  multiple  rounds,  each 
round  consisting  of  a  single  attack.  In  each  round,  one 
random  attacker  and  one  random  defender  are  chosen. 
When  the  attacker  attacks  a  defender,  the  defender 
loses  health  points  in  the  amount  equivalent  to  the 
attacker’s  attack  points.  For  example,  if  Allan  is  the 
attacker  and  Barry  is  the  defender,  Barry  would  lose  4 
health  points. 

The  fighters  continue  to  randomly  attack  and  defend 
in  subsequent  rounds  until  there  is  only  one  fighter 
left,  who  is  then  declared  the  winner.  A  fighter  is 
removed  from  the  battle  when  his  life  points  become 
zero  (or  less). 


\ 

Question:  Question:  Which  fighter  is 
most  likely  to  win  the  battle? 

Send  your  answer  to  puzzlor@gmail. 
com  by  December  15, 201 4. The  winner, 
chosen  randomly  from  correct  answers, 
will  receive  a  $25  Amazon  gift  card.  Past 
questions  can  be  found  at  puzzlor.com. 

John  Toczek  is  the  Sr.  Director  of  Decision  Support  and 
Analytics  for  Aramark  Corporation  in  the  Global  Operational 
Excellence  group.  He  earned  his  BSc.  in  Chemical  Engineer¬ 
ing  at  Drexel  University  (1 996)  and  his  MSc.  in  Operations 
Research  from  Virginia  Commonwealth  University  (2005). 
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MORS’  Young  Analysts 

THE  NEXT  GENERATION  OF  LEADERS 


The  future  of  operations  research 
and  the  national  security  community 
depends  on  new  analysts  taking  the 
helm.  MORS’  Young  Analyst  Initiative 
facilitates  this  process  by  providing 
paths  for  emerging  analysts  to  engage 
with  MORS  through  publishing, 
meeting  participation,  volunteering, 
mentorship  and  recognition. 

To  highlight  the  achievements,  interests 
and  aspirations  of  young  analysts,  we 
turn  the  spotlight  on  one  deserving 
individual  in  every  issue  of  Phalanx. 

To  learn  more  about  the  Young  Analyst 
Initiative,  connect  with  other  young 
analysts,  see  past  featured  analysts 
and  learn  how  you  can  nominate  a 
deserving  analyst,  please  visit 

WWW.MORS.ORG/YA 


LT  J.  Benjamin  Peterman,  USN 


When  did  you  join  MORS? 

I  joined  MORS  in  early  2014,  shortly  after 
starting  my  master  s  in  systems  analysis 
(MSA)  through  the  distance  learning  program 
at  the  Naval  Postgraduate  School. 

What  was  your  childhood  ambition? 

When  I  was  a  child,  there  was  no  shortage 
of  things  that  I  wanted  to  do  with  my  life. 
Astronaut,  competitive  athlete,  police  offi¬ 
cer,  and  pretty  much  any  other  thing  a  kid  dreams  of  being  were  all  on  the  list, 
yet  strangely,  I  did  not  strongly  consider  entering  the  military  until  my  senior 
year  in  high  school.  Six  years  later,  after  I  earned  my  wings  as  a  Navy  helicop¬ 
ter  pilot,  one  of  my  junior  high  school  teachers  said  to  me,  “I  always  knew 
you  wanted  to  be  a  pilot.”  Somehow,  I  had  gone  after  my  childhood  ambition 
without  even  realizing  that  it  had  been  my  dream  for  so  long. 


Why  did  you  become  an  operations  analyst? 

Like  many  of  the  things  that  have  shaped  who  I  am,  luck  played  a  significant 
role.  When  the  time  came  for  me  to  select  where  I  wanted  to  go  for  my  shore 
tour,  with  some  helpful  encouragement  and  advice  from  my  former  OIC,  I 
chose  to  head  to  Washington,  DC  and  join  the  N81  team  of  analysts.  Once 
here,  my  boss  and  mentor  encouraged  me  to  further  challenge  myself  and 
develop  my  skills  as  an  operations  analyst  by  pursuing  an  MSA,  and  it  has  been 
one  of  the  best  decisions  five  ever  made. 


Where  do  you  see  yourself  in  five  years? 

In  the  next  five  years,  I  look  forward  to  returning  to  flying  out  in  the  fleet,  with 
hopes  of  returning  to  an  operations  analysis  billet  sometime  after. 

How  has  your  MORS  membership  benefited  you?  What  do  you  value  most 
about  your  membership? 

My  MORS  membership  has  given  me  a  consistent  tie  to  the  greater  analytic 
community  through  channels  such  as  the  MORS  symposium  and  Phalanx. 

I  think  that  all  too  often,  we  get  scope  locked  on  our  immediate  work,  and 
it  becomes  easy  to  lose  touch  with  the  big  picture  and  the  world  around  us. 
MORS  helps  us  maintain  that  connection,  which,  to  me,  is  the  most  valuable 
aspect  of  my  membership. 
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31st  ISMOR 

Ted  Smyth,  FS 


attendees.  To  that  end  the  keynote  speaker,  Major  General 
Nick  A.W.  Pope,  CBE,  the  current  Director  of  General 
Capability  at  UK  Army  Headquarters,  shared  his  views  of 
the  global  environment  and  the  challenges  faced  by  the 
British  military.  Not  surprisingly,  General  Pope  identified 
many  challenges  that  are  shared  by  the  US  military  and 
its  allies.  Faced  with  significant  personnel  and  budgetary 
reductions  in  an  extremely  uncertain  threat  environment, 
British  forces  are  attempting  to  balance  and  reconcile 
current  operational  and  training  needs  with  projected 
future  requirements  and  capabilities.  General  Pope 
highlighted  the  reality  of  attempting  to  accomplish  these 
tasks  while  experiencing  an  increasing  set  of  military  roles 
and  missions  that  now  include  the  conduct  of  small-scale 
expeditionary  operations,  the  provision  of  homeland 
defense  and  security  support,  the  conduct  of  alliance 
reassurance  operations,  and  preparations  to  conduct 
divisional-level  warfare.  To  deal  with  these  challenges 
General  Pope  is  relying  extensively  upon  the  analytical 
community  for  their  support. 


Following  General  Pope’s  keynote  address,  other  briefings 
and  papers  addressed  a  wide  range  of  international 
defense  and  analytical  topics  to  include  the  need  to 
develop  an  agile,  networked-enabled  command-and- 
control  structure;  development 
of  a  method  to  estimate  the  risk 
caused  by  debris  resulting  from 
successful  missile  intercepts; 
methods  to  optimize  the  assignment 
of  military  personnel  to  training 
courses;  provision  of  an  analytical 
framework  for  operations  planning; 
modeling  human  reasoning  in 
time-constrained  environments; 
a  concept  for  development  of 
a  cyberconflict  campaign-level 
wargame;  development  of  a 
risk-based  method  to  assess 
investments  in  cybersecurity; 
analysis  of  asymmetries  in  air 
combat;  a  capability-led  approach 
to  collaborative 
4  technology  planning; 


The  31st  International  Symposium  of  Military 

Operational  Research  (ISMOR)  was  held  July  29- 
August  1, 2014,  at  Royal  Holloway,  a  constituent 
college  of  the  University  of  London  and  located 
approximately  19  miles  west  of  central  London 
in  Egham,  UK.  Officially  endorsed  by  the  UK’s 
Ministry  of  Defence,  ISMOR  was  founded  by  the  late 
Professor  Ronnie  Shephard  and  has  grown  into  the  world’s 
foremost  international  conference  on  defense-related 

operations  research.  This  year’s  attendees  numbered 
close  to  100  and  included  representatives 
from  Australia,  Canada,  Israel,  South 
Korea,  Singapore,  Sweden, 

Switzerland,  Turkey,  the 
UK,  and  the  USA. 

Whereas  the  31st 
ISMOR  purposely  chose 
not  to  focus  on  a  central 
theme,  the  presentations 
and  discussions  covered  a 
wide  variety  of  global,  military, 
and  national  security 
challenges  common 
to  virtually  all  of  the 
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inclusion  of  artificial  intelligence  algorithms  in  wargame 
simulations;  methods  to  measure  fleet  readiness;  an 
historical  analysis  of  the  relationship  between  energy  use 
and  military  capability;  and  the  impact  of  time  latency 
of  information  in  networked  operations.  Lunch  breaks 
featured  briefings  on  a  variety  of  topics  offered  in  a  poster 
session  format.  These  included  development  of  a  toolset 
to  assess  naval  platform  design;  a  method  to  estimate  the 
lifespan  of  major  defense  equipment;  an  historical  look 
back  at  30  years  of  operations  research  contributions; 
methods  to  improve  logistics  modeling;  and  an  overview 
of  initiatives  to  implement  national  security  through 
technology. 

Of  note  was  the  inclusion  in  the  31st  ISMOR  of  a  daylong 
session  devoted  to  Cornwallis  Group  issues  that  focused 
on  the  analysis  of  commercial  and  market  force  roles  in 
conflict  management  and  mitigation.  Recognizing  that 
private  companies,  Civil  Services  Organizations  (CSOs), 
military  services,  and  defense  agencies  increasingly 
partner  in  the  prevention,  settlement,  and  transformation 
of  conflicts,  Cornwallis  XIX  posed  the  question  as  to 
how  analysis  has  been  used  to  produce  a  blended 
solution  plan  of  action  to  improve  the  effectiveness  of  the 
overall  response.  As  such,  presentations  in  this  session 
included  papers  on:  balancing  the  mix  of  regular  and 
non  regular  forces;  the  impacts  of  embedded  operations 
analysts  in  military  and  civilian  organizations;  efforts 
and  methods  to  redeploy  materiel  from  the  Afghanistan 
theater  of  operations;  the  impact  of  contractor  support  on 
operational  risk;  and  the  use  of  value  engineering  (V E)  as  a 
method  to  reduce  the  cost  of  products  and  services. 

Evenings  at  ISMOR  proved  to  be  both  enjoyable 
and  informative.  Informal  presentations  provided  the 
opportunity  for  attendees  to  interact  socially  as  well  as  to 
receive  a  briefing  on  the  patterns  of  global  piracy,  where 


the  main  piracy  risks  reside,  and  a  brief  history  of  Chinese 
military  operations  research.  By  tradition  the  third  evening 
of  ISMOR  is  reserved  for  a  semi-formal  dinner,  followed 
by  the  Professor  Ronnie  Shephard  Memorial  Address. 
Selection  of  a  speaker  to  deliver  the  Shephard  Address 
is  reserved  for  a  notable  operations  analyst  who  has 
made  significant  contributions  to  the  state  of  operations 
analysis  on  either  a  national  and/or  international  scale.  The 
overwhelming  selection  for  this  year’s  honored  speaker 
role  was  Mr.  Gene  Visco,  FS,  known  to  all  MORSians  and 
the  greater  international  operations  research  community. 
As  is  his  norm,  Gene  delivered  a  humorous  and  eloquent 
address  in  which  he  shared  his  personal  reminiscences 
on  the  contributions  of  Professor  Shephard,  the  history 
of  how  operational  research  was  “invented”  and  crossed 
the  Atlantic  to  become  operations  research,  and  the 
challenges  that  still  remain  for  the  practitioners  of 
operations  research. 

The  31st  ISMOR  was  my  fourth  opportunity  to  engage 
with  colleagues  from  the  international  community.  As 
with  my  other  visits,  I  found  the  briefings  and  papers  to 
be  exceptional  in  both  topic  and  content.  As  first  time 
attendee  Bill  McDaniel  from  JHU/APL  noted,  “A  benefit 
of  this  type  of  international  forum  is  the  insight  gained 
on  similar  struggles  and  successes  in  military  analysis 
among  allied  countries.  For  instance,  it  seems  that  most 
countries’  military  planners  and  analysts  are  struggling 
with  the  complexity  of  the  operating  environment  and  are 
using  more  “soft  OR”  approaches,  including  qualitative 
measures,  to  supplement  their  analysis.” 

I  enthusiastically  recommend  that  if  you  have  the 
opportunity  to  attend  the  32nd  ISMOR  scheduled  for  late 
July  2015  that  you  do  so.  Briefings  from  the  31st  ISMOR 
as  well  as  information  on  the  32nd  ISMOR  are  posted  on 
the  ISMOR  website  (www.ismor.com). 
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The  Pseudo-Analytical  Agenda 

ARE  WE  WINNING? 

BY  B.  CRATCH  IT 


Location:  a  cave  in  an  ungoverned  region. 

Time:  evening,  after  a  dinner  of  captured  MREs. 
Attendees:  the  Revolutionary  Operations  and  Plans  Group 


Bill:  OK,  let’s  get  this  going.  How  did  we  do  today?  Who  wants  to  go  first? 

Bob:  I  can  cover  mobile  ops. 

Bill:  Please  do. 

Bob:  We  drove  around  five  villages  in  some  Humvees  we  stole.  We  timed  it  so  that  we  were  out  for  every  other 
hour  around  the  clock. 

Joe: . And  ...  ? 

Bob:  Well,  that  forced  them  to  keep  two  E-8C  JSTARS  orbits  going.  At  $82,800  a  flying  hour,  that’s  costing  them 
about  $1. 1M  a  day.  One  of  the  villages  is  near  the  coast,  so  they  have  to  keep  an  E-2C  up  -  that’s  another  $14,000 
per  flying  hour,  so  that’s  another  $336K  a  day. 

Pete:  Did  we  lose  anyone? 

Bob:  Well,  a  2-ship  of  F-15E’s  got  one  of  them,  but  they  dropped  10  GBU-39  Small  Diameter  Bombs  to  do  it.  The 
F-15E’s  were  flying  6-hour  sorties  at  $21,000  an  hour  and  the  bombs  cost  $214,000  each,  so  that  cost  them 
another  $2.4M. 

Bill:  Where  are  we  supposed  to  get  another  Humvee? 

Bob:  I  already  got  a  replacement.  Had  to  promise  the  guys  who  stole  it  a  couple  of  chickens,  though. 

Henry:  Look,  I  told  you  to  go  easy  on  giving  out  the  chickens!  What,  do  you  think  I’m  some  billionaire 
international  chicken  rancher? 

Bob:  Calm  down  -  we  didn’t  have  to  pay  the  chickens  we  promised  to  the  guy  who  got  bombed. 

Henry:  Well,  didn’t  his  family  want  them? 

Bob:  No,  we  just  told  them  the  guy  was  working  for  the  cause.  I  gave  them  some  flyers,  though.  They  were  mad 
enough  that  they  might  steal  another  Humvee  for  us. 

Pete:  OK,  enough.  How  about  air  defense? 
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Joe:  The  boys  managed  to  turn  on  that  radar  we  found.  We  have  no  idea  what  it  does,  but  we  fiddle  with  dials  and 
move  it  a  couple  of  times  a  day  We  figure  that  forces  them  to  maintain  two  E-3  Sentry  orbits  around  the  clock  at 
$14,200  an  hour,  plus  a  RC-135  U/V  RIVET  Joint  at  $23,200  an  hour.  So  that’s  another  $1.2M  a  day. 

Pete:  Don’t  they  shoot  at  you? 

Joe:  Well,  we  usually  wire  together  a  bunch  of  microwave  ovens  we  found  at  dump  and  turn  them  on  when  we 
leave.  We  did  get  them  to  drop  a  GBU-32  JDAM  on  one  pile  of  them,  but  that  only  cost  them  about  $30K. 

Pete:  We  need  to  do  better  there.  See  me  afterward.  How  about  fixed  targets? 

John:  Well,  I’ve  got  some  real  good  news. 

Pete:  What’s  that? 


John:  We  managed  to  get  the  power  on  in  that  old  fertilizer  factory,  and  used  those  trucks  you  got  us  to  run  back 
and  forth  for  a  couple  of  days.  And  guess  what  they  did? 

Pete:  I  give  up  -  what? 

John:  they  fired  17  BGM-109E  Tomahawk  cruise  missiles  at  the  building,  at  $1,120,000  each!  Plus  they  put  3 
MQ-9  Reaper  drones  up  for  12  hours  prior  to  watch  it,  at  $1000  a  flying  hour!  $19M! 

Pete:  Not  bad.  Did  they  get  any  of  the  trucks? 


John:  We  left  one  there,  but  it  threw  a  rod  and  we  couldn’t  get  it  running  anyway. 


Bob:  Well,  let  me  talk  to  them  -  I  like  that  stuff  they 
drink. 

Henry:  That’s  because  you’re  an  idiot  with  no  taste. 

Pete:  OK,  enough.  Not  a  bad  day  -  they  spent  at  least  $24M 
that  we  know  about.  You  guys  all  set  for  tomorrow? 

John:  All  except  Henry . 


Henry:  Easy  for  you  to  say,  but  I’ve  gotta  get  another 
truck!  Do  you  know  how  much  bad  liquor  I  have  to  drink 
with  those  guys  to  get  them  to  sell  me  one? 


Henry:  I’m  not  General  Motors  either!  What  am  I 
supposed  to  do  for  another  truck? 


Pete:  Relax,  that  was  one  of  those  Eastern  Bloc  things 
we  can  never  get  parts  for  anyway.  If  they  want  to 
shoot  it  with  a  $1M  cruise  missile,  that’s  fine  by  me. 


Henry:  I  still  don’t  know  where  I’m  going  to  get  more  chickens  ... ! 


IN  TOUCH 


Memb"Mi  estones 


Dr.  Jacqueline  R.  Henningsen  retired 
October  6  as  the  Director  of  Air  Force 
Studies  &  Analyses,  Assessments, 
and  Lessons  Learned.  Dr.  Henningsen 
is  a  MORS  Fellow  and  Past-President 
of  the  Society.  Look  for  a  special 
tribute  to  her  in  this  issue. 

Mr.  Kevin  Williams  has  been  named 
the  US  Air  Force  Director  of  Studies  & 
Analyses,  Assessments,  and  Lesson 
Learned.  As  such  he  become  the  Air 
Forces  senior  operations  analysts  and 
MORS  sponsor. 

Dr.  Priscilla  Glascow  has  retired  from 
the  MITRE  Corporation,  completing 
a  36-year  career  as  an  operations 
analyst.  Her  work  has  included 
being  the  DoD  expert  on  simulation 
credibility  assessment  at  the  Defense 
Modeling  and  Simulation  Office, 
retiring  as  a  Naval  Officer,  and  the 
last  17  years  at  MITRE.  Priscilla  has 
served  in  many  different  leadership 
capacities  with  MORS  over  the 
years,  including  twice  serving  on  the 


Board  of  Directors  and  as  Secretary- 
Treasurer  and  the  first  Vice  President 
for  Finance  and  Management.  She  is 
Fellow  of  the  Society.  Priscilla  intends 
to  enjoy  her  time  swimming,  working 
on  family  genealogy,  and  travelling. 

MORS  Directors  Aaron  Burciaga 
and  Harrison  Schramm  have 
been  invited  to  join  the  INFORMS 
Business  Analytics  .  Each  will  lead 
the  development  of  one  of  the 
eight  highly  selective  tacks  ranging 
from  Supply  Chain  Management  to 
Marketing  Analytics.  For  more  on  the 
INFORMS  Business  Analytics  effort, 
see  https://www.informs.org/Attend- 
a-Conference/Analytics-Conference. 

MORS  President 
Rafael  Matos 

and  Past 
President  Mike 
Kwinn  crossed 
paths  during  the 

Marine  Corps  Marathon,  when  Raf 
stopped  long  enough  to  visit  Mike’s 


The  Friends4Michael  Foundation 
booth.  So  far,  no  commitment  for  both 
to  run  the  race  together  next  year. 

MORS  Past 
President 

Terry 

McKearney 

recently  spent 
a  week  at 
sea  aboard 
the  USS  George  Washington  (CVN- 
73)  as  a  guest  of  his  daughter  LT 
Jennifer  Bridges.  The  Washington 
is  homeported  in  Japan  and  Jennifer 
is  the  Intelligence  Officer  (N2)  for 
Commander,  Destroyer  Squadron 
FIFTEEN  embarked  aboard  the  carrier. 
It  took  his  daughter  to  get  Terry,  a 
lifelong  Surface  Warfare  Officer,  to 
break  his  vow  never  to 
sail  on  a  “bird  farm.” 


Meet  Kevin  Williams:  New  Air  Force  Sponsor 

MORS  welcomes  Kevin  Williams  as  the  new  AF  sponsor.  He  became  the  Director  of 
Studies  and  Analyses,  Assessments  and  Lessons  Learned,  Headquarters  US  Air  Force, 

Washington,  D.C  AF/A  9,  replacing  Dr.  Jackie  Henningsen  in  early  October  (see  article  on 
page  2  of  this  issue  on  Ms.  Henningsen’s  retirement).  In  this  position  he  is  responsible 
for  the  development  of  Air  Force-wide  policy  and  guidance  that  ensures  defendable 
studies,  analyses,  assessments,  and  lessons  learned  processes  support  Air  Force 
leadership  decisions  to  enhance  current  and  future  warfighting  capabilities.  Prior  to 
joining  the  senior  executive  service,  Mr.  Williams  had  a  distinguished  Air  Force  career 
as  an  operational  fighter  pilot  flying  the  A-10,  F-16  and  F-1 1 1  and  as  a  staff  officer  with 
assignments  as  Director  of  the  US  Strategic  Command  Global  Innovation  and  Strategy 
Center;  Director  of  the  Air  Combat  Command  Commander’s  Action  Group;  Chief  of  the 
Information  Operations  Division  at  US  Northern  Command;  and  National  Defense  Fellow 
in  the  Secretary  of  Defense  Strategic  Studies  Group.  He  has  a  Master  of  Science  degree 
in  electrical  and  computer  engineering  from  the  University  of  Texas,  and  a  Master  of  Airpower  Art  and  Science  degree, 
from  the  School  of  Advanced  Airpower  Studies  at  Maxwell  AFB,  Alabama. 
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Let's  Think  Outside  the  Box 

So  We  Can  Matter  More 

to  Our  Customers 

John  Lawson  III,  Contractor,  US  Marine  Corps  Combat  Development  Command's  Analysis  Directorate, 
john.rJawson.ctr@usmc.mil 


Does  military  operations  research  (OR) 
matter  as  much  as  it  should  ...  or 
could? 

The  Marine  Corps  Modeling  and 
Simulation  Management  Office 
recently  sent  me  to  the  2014  Game 
Developers  Conference  (GDC),  and  no 
matter  what  the  topic  of  a  particular 
session  was,  there  was  a  good 
chance  I  was  going  to  hear  the  game 
industry  guys  crow  about  the  value  of 
“analytics”  (they  never  say  “OR,”  and 
they  rarely  say  “analysis”). 

If  you  thought  the  conference  would 
emphasize  things  like  killer  graphics 
in  a  new  Batman  video  game,  you 
thought  wrong.  This  was  a  conference 
with  more  than  20,000  attendees,  and 
even  though  precious  few  of  them 
were  analysts,  they  couldn’t  help  but 
praise  “analytics.” 

Are  the  customers  of  military  OR 
that  enthusiastic?  If  they  don’t  value 
analysis  as  much  as  we  think  they 
should,  are  there  things  we  should  be 
doing  to  change  that? 

Let’s  think  outside  the  military  analysis 
box  for  a  bit. 

Admittedly,  there  are  huge  differences 
between  the  game  industry  and  the 
Department  of  Defense  (DoD).  Their 
business  involves  the  PlayStation, 
the  Xbox,  and  so  on.  Ours  involves 


bombs,  bullets,  and  so  forth.  Their 
ultimate  metric  is  profit.  Ours  is 
“victory”  (a  term  that’s  often  the 
subject  of  fierce  debate).  Sure,  there 
are  many  differences  between  them 
and  us.  But  thinking  about  what  they 
do  is  a  great  way  for  us  to  get  outside 
the  box  for  a  bit. 

The  gaming  industry’s  revenue  is  well 
over  $50  billion  annually.  The  industry 
deserves  at  least  a  little  bit  of  our 
attention  if  for  no  other  reason  than 
it’s  successful.  Over  the  past  1 0  or 
15  years,  it  has  improved  in  dramatic 
and  innovative  ways  (and  hasn’t 
suffered  frustrations  equivalent  to  our 
Afghanistan  and  Iraq  experiences). 

There  are  many  facets  of  the  gaming 
industry:  programming,  artificial 
intelligence,  marketing,  network 
management,  quality  assurance, 
etc.  Talent  from  all  segments  of 
the  industry  attends  the  GDC,  and 
if  a  professional  conference  is  an 
indication  of  an  industry’s  health, 
game  development  is  doing  quite  well. 
When  I  was  an  Air  Force  contractor 
and  attended  the  2004  GDC,  there 
were  about  10,000  attendees.  This 
year,  as  a  Marine  contractor,  I  was 
among  more  than  twice  that  many 
people. 

So,  let’s  look  at  some  ways  analysts 
make  themselves  invaluable  to  the 
gaming  industry. 


Analysis  Is  Frequently 
Organic  (It's  Rarely  an 
Appendage) 

For  many  game  development  teams, 
analysis  is  one  more  part  of  an  organic 
whole.  It’s  not  an  appendage  or  an 
outsider.  Analysis  isn’t  like  a  doctor 
you  go  to  when  you’re  sick,  nor  is  it 
like  a  lawyer  you  go  to  when  you’ve 
been  sued.  It’s  like  the  offensive  line, 
which  blocks  for  the  quarterback, 
blocks  for  the  halfback,  blocks  for  the 
fullback,  and  blocks  when  a  receiver 
runs  a  reverse. 

Because  analysis  is  integral,  rather 
than  external,  it  is  intimately  aware 
of  what  things  need  to  be  tracked 
and  how  those  things  need  to  be 
tracked.  Analysis  teams  know  how 
they  need  to  segment  and  identify 
customers.  They  know  how  to 
distinguish  between  the  way  some 
users  play  a  game  and  the  way  other 
users  play  the  same  game.  They  know 
how  to  make  connections  between 
customers’  gaming  habits  and  their 
spending  habits. 

When  it’s  time  to  make  a  new  game, 
analysts  are  often  involved  at  the  outset. 
They  establish  metrics  and  ensure 
appropriate  data  collection  for  those 
metrics.  Many  game  development 
projects  are  born  with  metrics  in  mind. 
It’s  not  unusual  to  identify  10,000 
metrics  when  a  game  is  in  its  embryonic 
stages,  according  to  one  speaker. 
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And  metrics  aren’t  a  bunch  of 
things  for  someone  to  ponder  while 
holding  a  pipe  in  one  hand  and  a 
glass  of  Chablis  in  the  other.  Many 
are  real-time  metrics  that  feed  the 
maintenance  and  operation  of  the 
game  during  its  life. 

Analysis  Helps  the  Project, 
the  Project  Helps  Analysis 

For  many  games,  especially  the 
bigger  ones,  almost  everyone  involved 
is  actively  using  analysis.  And  the 
analysts  are  working  with  almost  all  of 
the  team’s  groups  to  ensure  that  each 
group  has  the  analysis  it  needs  to 
make  the  game  successful. 

Naturally,  analysts  want  data  that 
supports  metrics  so  their  analysis 
will  be  good.  And  everyone  in  the 
game  business  wants  good  analysis. 
Marketers  want  good  analysis  so 
they  can  make  the  game  popular. 

Folks  in  customer  service  want 
good  analysis  so  they  can  prevent 
complaints  or  handle  complaints 
swiftly  and  efficiently.  Folks  in  quality 
assurance  want  good  analysis  so 
they  can  address  problems  before 
customers  even  notice  them. 
Designers  want  good  analysis  so 
they  can  tell  which  aspects  of  a 
game  hold  the  customer’s  attention 
and  which  aspects  of  a  game  cause 
the  customer  to  bail  out.  Developers 
want  good  analysis  so  they  can 
push  out  good  updates  and,  if  the 
game  is  successful,  make  version  2 
better  than  version  1 .  Finally,  network 
managers  want  good  analysis  so  they 
can  allocate  servers  wisely. 


And  chew  on  this:  A  decade  ago, 
at  the  2004  GDC,  very  few  voices 
had  much  to  say  about  analysis.  In 
other  words,  analysis  has  earned 
a  central  seat  at  the  table.  It  didn’t 
inherit  a  seat  at  the  table.  It  didn’t  get 
by  on  tradition  or  old  achievements. 

It  powered  its  way  to  a  prominent 
position  in  less  than  10  years. 


The  More  Analysis  Gets 
Involved,  the  More  Analysis 
Gets  Involved 

The  more  the  game  industry  uses 
analysis,  the  more  new  ways  the 
industry  discovers  to  use  analysis. 
Analysis  tells  game  developers 
who  is  playing  the  game  and  when 
they’re  playing  the  game.  Then 
analysis  identifies  users  and  creates 
segmentation  schemes  to  bin 
users.  Then  analysis  looks  at  the 
differences  between  how  people  in 
Groups  A,  B,  C,  etc.,  play  the  game. 
Experiments  with  centralized  control 
are  also  possible.  These  approaches 
lead  to  immediate  payoffs  in  games 
that  include  advertising  or  selling 
accessories.  And  these  approaches 
lead  to  long-term  payoffs  during  the 
development  of  subsequent  games. 


It’s  not  surprising  that  Amazon  and 
Google  were  heavyweight  sponsors 
of  the  2014  GDC,  even  though  they 
were  practically  invisible  at  the  2004 
GDC.  Those  two  companies  know 
how  to  crunch  data.  Clearly,  they 
have  identified  the  gaming  industry 
as  a  growth  area  for  businesses  that 
Excel  (pun  intended)  at  analyzing 
phenomena  in  high-tech  settings. 


One  of  the  most  interesting  things 
about  the  2014  GDC  was  the  fact 
that  game  developers  might  talk  for 
several  hours  without  uttering  the 
name  of  a  single  game.  They  had  so 
much  to  say  about  “analytics”  that 
they  often  forgot  to  mention  “Madden 
NFL”  or  “Call  of  Duty.” 


The  Emphasis  Is  on  Now 
and  the  Future 

While  I’m  suggesting  that  we  should 
contemplate  how  the  gaming 
industry  uses  analysis,  I  would  like 
to  stretch  this  trip  outside  the  box 
just  a  little  bit  further.  Let’s  not  only 
think  about  how  game  developers 


use  analysis.  Let’s  think  about  game 
development  in  general. 

Thanks  to  math,  computer  science, 
engineering,  physics,  and  the  other 
subjects  that  tend  to  define  military 
analysts,  we  have  a  tendency  to 
regard  ourselves  as  scientists. 

Science  is  usually  associated  with 
progress.  Are  we  committed  to  the 
future?  Are  we  even  committed  to  the 
present  day’s  state  of  the  art?  Are  we 
a  little  too  old-school? 

In  the  gaming  industry,  the  past  is 
in  the  rearview  mirror.  I’m  47  years 
old.  Of  the  roughly  24,000  people  at 
the  2014  GDC,  I  only  saw  about  30 
people  who  were  definitely  older  than 
I  am.  When  I  go  to  a  DoD  meeting, 

I’m  usually  right  around  the  median 
age.  In  DoD,  we’re  very  credential 
oriented.  Credentials,  of  course,  are 
something  you  accumulated  in  the 
past.  The  gaming  industry,  however, 
wants  to  know  what  you’re  doing 
right  now.  The  gaming  industry 
openly  wonders  what  the  expiration 
date  is  for  academic  preparation. 
During  the  conference,  one  professor 
of  game  development  said  that 
curricula  are  constantly  in  flux 
because  “what  I  knew  six  months 
ago  may  be  out  of  date  today.”  As  a 
result,  the  industry  emphasizes  what 
potential  hires  have  produced  rather 
than  what  they’ve  studied. 

Whether  it’s  DOD  analysis,  DOD 
Modelling  and  Simulation  (M&S), 
or  DOD  in  general,  it’s  pretty  easy 
for  us  to  go  to  a  meeting  and  see 
PowerPoint  presentations  that 
resemble  the  ones  we  saw  in  2004. 

In  the  gaming  industry,  2004  might  as 
well  be  the  era  of  biplanes.  In  the  past 
1 0  years,  the  industry  has  seen  PC 
games  go  from  central  to  marginal; 
phone  games  go  from  marginal 
to  central;  tablet  games  go  from 
unimagined  to  very  popular;  networks 
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for  console  games  go  from  new  and 
promising  to  wildly  successful  (tens 
of  millions  of  users);  game  storage  go 
from  discs  to  downloads  and  clouds. 
And  it’s  easy  to  imagine  more  change 
in  the  near  future.  No  one  at  GDC 
2014  seemed  quite  sure  how  Google 
Glass  will  enter  the  gaming  mix,  but 
there  was  plenty  of  agreement  that 
such  hardware  will  soon  join  the 
gaming  party. 

Conclusion  - 

Yes,  I  know  that  many  aspects  of 
the  gaming  industry  don’t  apply  to 
DOD  in  general  or  to  DOD  analysis 
in  particular.  There  are  numerous 
reasons  to  look  at  the  gaming  industry 
and  say,  “That  doesn’t  apply  to  us.” 
Maybe  a  lot  of  it  doesn’t.  But  before 
we  climb  back  into  the  box  and  stick 
with  business  as  usual,  let’s  red- 
team  how  we  do  business.  Is  the 
national  security  analytical  community 
structurally  optimized  to  deliver  the 
best  service?  Are  our  customers 
greedily  consuming  what  we  give 
them,  and  are  they  craving  more?  Are 
we,  as  a  science-oriented  profession, 
on  the  cutting  edge?  (This  PC  I’m 
using  to  type  this  article  certainly  isn’t.) 

As  it  stands,  we  as  analysts  are 
doing  a  good  job.  But  we’re  in  a 
profession  that  revolves  around  the 
word  “optimize.”  Optimal  is  more  than 
just  good.  Optimal  is  the  best.  Are  we 
doing  the  best  we  can? 

I  want  to  be  clear:  I’m  not  talking 
about  analytical  techniques.  I’m  talking 
about  increasing  the  opportunities  we 
have  to  be  relevant.  The  more  involved 
we  can  be  with  customers  and 
potential  customers,  the  better.  This 
is  outside-the-box  thinking.  If  there’s 
a  rap  on  the  analytical  personality, 


it’s  that  we  tend  to  be  introverts  - 
introverts  who  believe  they  can  be 
quiet  because  the  facts  will  speak  for 
themselves.  We  need  to  speak  more 
for  analysis,  promote  analysis,  and 
connect  analysis  more  strongly  to 
DOD  activities. 

Whether  I’ve  put  my  finger  on  a  minor 
problem  (going  from  A-  to  A+)  or  a 
major  problem  (going  from  C  to  A), 
what  I’ve  attempted  is  a  first  stab  at 
framing  a  problem.  Obviously,  you 
can’t  have  a  solution  until  you  know 
that  the  problem  exists.  And  you’ve  got 
to  be  able  to  describe  the  problem. 

Having  said  that,  here  are  some 
examples  of  possible  opportunities 
for  structural  optimization  in  our 
profession: 

Sometimes  military  OR  adheres 
to  a  study  schedule  that  no  longer 
addresses  the  customer’s  needs  by 
the  time  work  is  done.  This  is  like 
having  the  perfect  game  plan  for 
February’s  Super  Bowl,  but  handing  it 
to  the  coach  in  April. 

We  all  have  stories  about  how  we 
came  up  with  the  right  answer,  but  it 
was  disregarded.  This  is  a  problem 
that  requires  fixing,  not  complaining. 

We  often  have  trouble  getting  the 
data  we  need,  even  when  we  know 
the  data  exists.  Doctors  don’t  have 
trouble  getting  X-rays,  accountants 
don’t  have  trouble  getting  tax  records, 
so  why  do  we  have  such  problems? 

Sometimes  the  customer  isn’t 
committed  to  strategic  goals,  and 
sometimes  the  customer’s  subject 
of  interest  is  in  flux,  yet  we  lock 
ourselves  into  an  approach  that  is 


slow  and  likely  to  fall  out  of  synch 
with  both  the  customer  and  the  topic. 
A  lot  of  work  on  irregular  warfare  fell 
into  this  trap. 

Sometimes  we’re  more  interested  in 
an  advisory  role  than  a  participatory 
role,  which  makes  it  harder  to 
contribute  because  we  are  outsiders. 
This  may  make  it  easier  to  be 
objective,  but  it  makes  it  more  likely 
that  key  players  will  treat  us  as  strap¬ 
hangers  or  nuisances. 


I  felt  a  little  envy  for  the  analysts 
who  serve  the  gaming  industry.  Their 
customers  are  addicted  to  them. 

Let’s  get  our  customers  addicted  to  us. 


About  the  Ai 


John  Lawson  III  is  a  contractor  who 


works  for  Group  W  and  serves  as  an 
analyst  for  the  Marine  Corps  Combat 
Development  Command’s  Analysis 
Directorate.  Before  that,  he  was  a 
contractor  serving  several  Air  Force 
entities  as  an  analyst.  In  the  1990s, 


he  was  a  newspaper  reporter,  mainly 
for  the  Tampa  Tribune;  he  is  also  the 
author  of  Tom  Landry  and  Bill  Walsh: 
How  Two  Coaching  Legends  Took 
Championship  Football  from  the 
Packer  Sweep  to  Brady  vs.  Manning. 
Lawson  served  in  the  Marine  Corps 
Reserve  for  nine  years  and  reached 
the  rank  of  staff  sergeant.  He  has  a 
BS  in  mechanical  engineering  from 
the  University  of  Maryland;  a  BA  from 
Washington  &  Lee  University  for  a 
double  major  in  history  and  English; 
and  an  MA  from  the  University  of 
Florida  in  mass  communications. 
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Dr.  George  Akst,  Marine  Corps  Combat  Development  Command,  george.akst@usmc.mil 


About  five  years  ago,  I  was 
invited  to  participate  in 
a  seminar  hosted  by  the 
Johns  Hopkins  University 
Applied  Physics  Lab  on 
Unrestricted  Warfare  (Akst, 
2009).  I  believe  that  event  was  a 
turning  point  in  my  thinking  about 
many  of  the  types  of  nontraditional 
analyses  that  are  addressed  in 
articles  in  this  Phalanx  edition  by 
Dr.  Yuna  Huh  Wong  and  Mr.  John  R. 
Lawson.  Interestingly,  this  symposium 
predated  the  Marine  Corps’  foray 
into  the  non-traditional  analysis  that 
has  come  to  be  known  as  the  Joint 
Irregular  Warfare  Analytic  Baseline,  or 
JIWAB.  Although  I  will  not  repeat  the 
details  of  that  Hopkins’presentation 
here— I  refer  the  reader  to  the 
documentation  in  the  reference— I 
will  point  out  the  five  principles  that  I 
used  in  that  presentation,  known  as 
Akst’s  Axioms: 

•  It  ain’t  your  father’s  enemy 
•  Focus  on  the  model .  .  .  not  the 
simulation 

•  It’s  the  data,  stupid 
•  Processes  may  themselves  be 
changing  from  deterministic  to 
stochastic 

•  Most  (DoD)  analyses  are  not 
predictive 


You’ll  note  the  plagiarism  in  the  title  of 
this  article  from  my  first  axiom,  and  for 
those  faithful  readers  of  Phalanx,  you’ll 
note  other  acts  of  plagiarism  as  well. 

(I  have  no  issues  with  plagiary,  at  least 
when  the  target  is  myself.  Others  may 
say  I  have  had  no  new  ideas  in  years.) 

The  idea  behind  this  article  is  to 
acquaint  the  reader  to  some  potentially 
new  ways  of  thinking  about  analysis, 
and  to  introduce  the  two  associated 
articles  in  this  edition  by  Wong  and 
Lawson.  Many  traditional  defense 
analysts  have  had  similar  experiences 
in  the  field  of  operations  research 
(OR),  beginning  with  some  sort  of 
formal  training  (often  a  master’s  in 
OR),  followed  by  a  series  of  jobs 
training  the  analyst  to  use  his  or  her 
skills  to  address  important  operational 
questions.  I  have  often  seen  these 
problems  addressed  by  going  through 
one’s  “catalog”  of  analytical  tools 
and  choosing  the  one  that  seems  the 
most  appropriate.  However,  with  the 
types  of  problems  we  are  often  facing 
today,  we  probably  need  to  open  the 
aperture  wider  than  simply  the  toolset 
that  our  master’s  degree  armed  us  with. 
To  quote  a  cliche,  we  need  to  think 
outside  the  box.  I  often  use  the  9-dot 
matrix  problem  to  illustrate  how  most 
of  us  have  difficulty  thinking  outside 
the  box  (Figure  1).  In  this  problem,  the 
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subject  is  asked  to  connect  all  nine 
dots  with  a  total  of  four  straight  lines, 
without  lifting  the  pencil  from  the  paper 
(e.g.,  the  beginning  of  each  subsequent 
line  is  the  end  of  the  previous  one). 

Most  people  agonize  by  drawing,  or 
visualizing,  lines  that  stay  within  the 
boundaries  of  the  original  box,  but 
the  problem  cannot  be  solved  without 
thinking  (going)  outside  the  box. 

So,  back  to  the  title  of  this  article. 
Often,  the  analytical  problems  we 
are  seeing  lately  do  not  fit  nicely  into 
the  toolbox  of  existing  techniques. 
More  and  more,  we  need  to  reach 
outside  this  toolbox  and  develop  new 
approaches.  For  some  of  us,  this  may 
be  pushing  ourselves  outside  of  our 
comfort  zones,  and  perhaps  even 
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stretching  the  definitions  of  defense 
analysis  techniques.  So,  although  it 
may  not  be  your  father’s  OR,  perhaps 
it  is  closer  to  your  grandfather’s  OR. 
Recall  all  of  those  pioneers  back  at  the 
dawn  of  the  defense  OR  era  during 
World  War  II;  none  of  them  had  formal 
training  in  OR,  or  a  handy  toolbox  to 
rely  upon.  They  had  to  think  about 
problems  and  develop  appropriate 
techniques  on  the  spot,  often  in 
multidisciplinary  environments. 

As  I  move  into  the  twilight  of  my  career, 
and  watch  many  of  my  colleagues 
retiring  to  lives  of  leisure,  I  realize  that 
often  words  of  wisdom  from  a  waning 
generation  may  not  ring  true  with  our 
younger  readers.  With  this  in  mind,  let 
me  present  an  example  that  may  be 
more  meaningful  to  this  generation. 

For  more  than  a  century,  baseball 
managers,  owners,  and  scouts  relied 
on  their  tried-and-true  toolbox  of 
statistics  to  craft  winning  teams:  runs 
batted  in,  batting  average,  stolen 
bases,  etc.  Then  came  the  revolution 
immortalized  in  Moneyball  (Lewis, 
2003).  Closer  scrutiny  to  success 
indicators  showed  that  there  were 
better  (and  cheaper)  indicators  of 
success,  such  as  on-base  percentage 
and  slugging  percentage.  And  thus 
began  a  revolution  that  changed  the 
way  that  an  entire  profession  began  to 
think  about  the  game.  So,  I  encourage 
all  of  you  young  analysts  out  there  to 
write  the  next  version  of  Moneyball 
for  defense  analysts,  with  the  focus 
on  how  we  can  think  about  the  tools 
and  techniques  of  our  profession  in  a 
manner  that  is  perhaps  more  focused 
on  the  21st  century. 

I’ll  conclude  with  just  a  brief  word 
about  the  work  of  two  of  our  bright 
young  analysts  who  have  tried  to 
think  outside  the  box  with  their 
articles  in  this  edition  of  Phalanx. 
Wong’s  article  does  a  good  job  of 
addressing  the  history  of  OR,  and 


Figure  1 . 9-Dot  Matrix  Problem 


the  potential  role  of  “soft  OR,”  and 
the  more  multidisciplinary  nature  of 
our  current  techniques.  Wong  also 
discusses  the  JIWAB  approach, 
which  is  but  one  example  of  how  we 
have  stretched  the  boundaries  of  OR. 
As  anyone  familiar  with  baseball’s 
Moneyball  revolution  knows,  there 
were  diehard  baseball  geeks  who 
had  devised  new  and  better  ways 
to  understand  the  game.  But  for  all 
their  intellectual  horsepower,  these 
geeks  did  not  matter  until  someone 
formally  incorporated  their  thinking 
into  the  day-to-day  operations  of 
a  baseball  team.  Though  Lawson 
doesn’t  mention  Moneyball,  to  put 
his  argument  into  Moneyball  terms, 
he  argues  that  whereas  the  military 
values  OR  more  than  baseball  used 
to  value  its  geeks,  the  military  doesn’t 
value  OR  as  much  as  baseball 
currently  values  its  geeks.  He  says 
we  need  to  change  that. 

References 

1.  Akst,  G.  2009.  “Analysis  Support 
for  Unrestricted  Warfare,” 
Proceedings  on  Combating  the 
Unrestricted  Warfare  Threat, 
Johns  Hopkins  University,  Applied 
Physics  lab  (APL),  417-423; 
http://www.jhuapl.edu/urw_ 
symposium/Proceedings/2009/ 
Book/2009URWBook.pdf. 

2.  Lewis,  M.  2003.  Moneyball:  The 
Art  of  Winning  an  Unfair  Game, 
W.W.  Norton.  Note:  The  movie 


Moneyball  starring  Brad  Pitt  was 
released  in  201 1 . 

About  the  Author 

Dr.  George  Akst  is  the  senior  analyst 
for  the  Marine  Corps  Combat 
Development  Command  (MCCDC), 
Ouantico,  Virginia.  Akst  graduated 
cum  laude  from  the  City  College  of 
New  York  and  received  his  PhD  in 
mathematics  from  the  University  of 
Illinois  in  1974.  He  was  professor  of 
mathematics  at  New  Mexico  State 
University  and  the  California  State 
University  at  San  Bernardino  prior  to 
joining  the  Center  for  Naval  Analyses 
(CNA),  where  for  1 9  years  he  was 
a  research  operations  analyst  and 
study  director  for  numerous  projects, 
including  Cost  and  Operational 
Effectiveness  Analyses  (COEA)  on  Sea- 
Based  Tactical  Ballistic  Missile  Defense, 
Tomahawk  Baseline  Improvement 
Program,  Unmanned  Aerial  Vehicles, 
and  Advanced  Amphibious  Assault 
Vehicle.  He  directed  analyses  of 
Marine  Corps  Ground  Operations  in 
Desert  Storm,  and  led  all  of  CNA’s 
efforts  to  support  both  the  Navy  and 
Marine  Corps  in  the  Ouadrennial 
Defense  Review.  He  also  served  as 
the  scientific  analyst  to  Headquarters, 
Marine  Corps  (Plans,  Policies,  and 
Operations  [PP&O]  and  Programs  and 
Resources  [P&R]),  MCCDC,  and  the 
Navy  Staff  (Director  of  Assessments 
(N-81));  and  as  the  field  representative 
to  the  Fleet  Marine  Force,  Pacific. 


PHALANX  -  DECEMBER  201  4  57 


Visual 


Opportunity 
at  your  fingertips. 


The  answers  you  need,  the  possibilities  you  seek  -  they're 
all  in  your  data.  SAS®  helps  you  quickly  see  through  the 
complexity  and  find  hidden  patterns,  trends,  key 
relationships  and  potential  outcomes.  Then  easily  share 
your  insights  in  dynamic,  interactive  reports. 


Try  Visual  Analytics  and  see  for  yourself 

sas.com/VAdemo 


3d  trademarks^Kademarks  of  SAS  Institute-lnc.  in  thffUSAfand  other  countries.  ®  indicates  USA  registration.  Other  brand  and  product  names  are  trademarks  of  their  respective  companies.  ©  2014  SAS  Institute  Inc.  All  rights  reserved.  SI  21 657US.0414 


